May 05 01:30:06.708304 np0000171612 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]: we strongly recommend against using it for new projects.
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]: If you are already using Eventlet, we recommend migrating to a different
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]: framework.  For more detail see
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
May 05 01:30:07.044921 np0000171612 neutron-ovn-maintenance-worker[104604]:   import eventlet
May 05 01:30:08.077538 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.common.config [-] Logging enabled!
May 05 01:30:08.078113 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev181
May 05 01:30:08.078113 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=104604) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
May 05 01:30:08.078113 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
May 05 01:30:08.078113 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=104604) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 05 01:30:08.078331 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=104604) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 05 01:30:08.078331 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=104604) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 05 01:30:08.078331 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=104604) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 05 01:30:08.078550 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=104604) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 05 01:30:08.078550 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [-] Loading core plugin: ml2
May 05 01:30:08.358538 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.358667 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.358667 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.359363 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.359363 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.359363 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.393584 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.393710 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.393710 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.491947 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.492117 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.492117 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.546501 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.546600 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.546673 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.556279 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.556406 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.556476 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.604335 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7c498e3fdda0> agent after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.604492 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7c498e3fdda0> agent after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.604492 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x7c498e3fde40> segment precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.604602 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x7c498e3fdee0> network precommit_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.604903 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x7c498e3ff100> segment before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665435 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> network before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665554 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x7c498e3df600> agent before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665615 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> network before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665682 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> port before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665785 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> port before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665853 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> segment precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665921 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> segment precommit_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.665989 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> segment after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666069 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> segment after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666069 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> network after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666193 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> network precommit_delete 0, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666253 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> port provisioning_complete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666320 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666508 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666508 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> agent after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666508 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> subnet after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666676 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> subnet precommit_delete 0, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666676 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666827 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666827 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> segment_host_mapping after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.666951 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> agent after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.667010 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.667067 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.667139 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c498e06d850>> rbac-policy before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.667514 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:08.667605 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:08.667605 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:08.667711 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:08.667711 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:08.667840 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:08.668017 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
May 05 01:30:08.668877 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
May 05 01:30:08.669619 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
May 05 01:30:08.670228 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
May 05 01:30:08.670332 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
May 05 01:30:08.671006 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
May 05 01:30:08.671006 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
May 05 01:30:08.671006 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
May 05 01:30:08.671899 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
May 05 01:30:08.671899 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
May 05 01:30:08.671899 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
May 05 01:30:08.710513 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
May 05 01:30:08.710513 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
May 05 01:30:08.710513 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
May 05 01:30:08.711114 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7c498deaeb10>> floatingip before_response 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.711192 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7c498deaeb10>> port before_response 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:08.730348 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
May 05 01:30:08.730348 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
May 05 01:30:08.730537 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
May 05 01:30:08.730537 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
May 05 01:30:08.778157 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO dbcounter [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Registered counter for database neutron
May 05 01:30:08.795065 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_db.sqlalchemy.engines [None req-cf0bf223-a096-42b5-b532-92698118373a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104604) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
May 05 01:30:09.209132 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG dbcounter [-] [104604] Writer thread running {{(pid=104604) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 05 01:30:09.394395 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Initializing driver for type 'local'
May 05 01:30:09.394613 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Initializing driver for type 'vlan'
May 05 01:30:09.508940 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-cf0bf223-a096-42b5-b532-92698118373a None None] VlanTypeDriver initialization complete
May 05 01:30:09.508940 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Initializing extension driver 'port_security'
May 05 01:30:09.508940 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.extensions.port_security [None req-cf0bf223-a096-42b5-b532-92698118373a None None] PortSecurityExtensionDriver initialization complete
May 05 01:30:09.508940 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Initializing extension driver 'qos'
May 05 01:30:09.509328 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.plugins.ml2.extensions.qos [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QosExtensionDriver initialization complete {{(pid=104604) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
May 05 01:30:09.509328 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Initializing mechanism driver 'ovn'
May 05 01:30:09.509328 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Starting OVNMechanismDriver
May 05 01:30:09.510626 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> process before_spawn 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.510731 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> process after_init 55550000, True {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.510836 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> segment after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.510901 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> segment after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.510956 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> segment after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511085 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> allowed_address_pair before_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511170 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511234 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511303 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511386 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group precommit_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511449 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511547 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group_rule precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511627 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group_rule after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511696 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> security_group_rule before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511815 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> address_group after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511881 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> address_group after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.511970 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c498df44d40>> address_group after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.512126 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x7c498dbf0200>> qos_plugin after_init 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.512345 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x7c498d1d8da0>> trunk_plugin after_init 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.512492 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x7c498d1d8dd0>> logging-plugin after_init 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.512834 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.services.logapi.drivers.ovn.driver [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVN logging driver registered
May 05 01:30:09.514028 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.plugins.ml2.plugin [None req-cf0bf223-a096-42b5-b532-92698118373a None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=104604) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 05 01:30:09.514214 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498dde6f00>> port after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514288 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498dde6f00>> port after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514372 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498dde6f00>> port after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514458 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d9040>> subnet after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514523 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d9040>> subnet after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514611 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d9040>> subnet after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514677 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8fb0>> network after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514790 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8fb0>> network after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514856 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8fb0>> network after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.514935 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f50>> security_group after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515003 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f50>> security_group after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515084 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f50>> security_group after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515148 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f20>> security_group_rule after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515216 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f20>> security_group_rule after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515492 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f20>> security_group_rule after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515557 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f80>> address_group after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515801 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f80>> address_group after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515801 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c498d1d8f80>> address_group after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.515801 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ML2 OVO RPC backend initialized. {{(pid=104604) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
May 05 01:30:09.516626 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.quota [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7c498dfaee70>.
May 05 01:30:09.516718 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.plugin [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Modular L2 Plugin initialization complete
May 05 01:30:09.516822 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading services supported by the core plugin {{(pid=104604) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
May 05 01:30:09.516894 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.plugins.ml2.managers [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Got port-security extension from driver 'port_security'
May 05 01:30:09.517151 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=104604) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
May 05 01:30:09.517317 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: ovn-router
May 05 01:30:09.524283 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c498d1da870>> router before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524378 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x7c498e05ae80> port after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524483 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x7c498e05ad40> port after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524550 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x7c498e05ade0> subnet after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524643 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x7c498e05af20> subnetpool_address_scope after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524706 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7c498d1b74c0> port before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524832 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7c498d1b74c0> port after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.524929 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c498d1da870>> port precommit_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.525022 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x7c498e03ede0> port before_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.525103 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x7c498e03ed40> port before_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.525187 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c498d1da870>> router precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.525304 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c498d1da870>> router precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.525386 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c498d1da870>> router precommit_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.525474 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x7c498d1b7420> subnet after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.526059 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Tracking information for resource: router configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:09.526137 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.quota.resource_registry [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Tracking information for resource: floatingip configured {{(pid=104604) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 05 01:30:09.526203 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.services.ovn_l3.plugin [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Starting OVNL3RouterPlugin
May 05 01:30:09.526397 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c498d19a120>> port_forwarding_plugin after_init 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.526552 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c498d1dac60>> router before_create 55550100, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.526630 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c498d1dac60>> router precommit_delete 55550100, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.526713 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c498d1dac60>> router precommit_create 55550100, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.526822 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c498d1dac60>> router precommit_update 55550100, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.529296 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.services.provider_configuration [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Service providers = [] {{(pid=104604) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
May 05 01:30:09.529384 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.db.servicetype_db [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=104604) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}}
May 05 01:30:09.530403 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router precommit_create 55549900, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530479 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router_interface after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530537 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> floatingip after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530638 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> floatingip precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530710 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> floatingip precommit_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530821 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> floatingip after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530888 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> floatingip after_status_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.530972 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> floatingip after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531048 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> port_forwarding after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531128 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router_interface after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531211 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router after_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531294 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531371 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router after_delete 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531455 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c498d1dbe60>> router after_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531544 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.services.service_base [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=104604) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
May 05 01:30:09.531624 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c498d1da870>> process after_init 55550001, True {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.531719 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.531836 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: auto_allocate
May 05 01:30:09.532511 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7c498d1c1760> network precommit_update 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.532617 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron_lib.callbacks.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7c498d1c1760> network precommit_create 55550000, False {{(pid=104604) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.532783 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.532864 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: tag
May 05 01:30:09.535196 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.535328 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: timestamp
May 05 01:30:09.536356 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.536420 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: network_ip_availability
May 05 01:30:09.537631 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.537702 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: flavors
May 05 01:30:09.538454 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.538557 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading Plugin: revisions
May 05 01:30:09.539337 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.manager [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=104604) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 05 01:30:09.539440 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_concurrency.lockutils [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.461s {{(pid=104604) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 05 01:30:09.539588 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=104604) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
May 05 01:30:09.539658 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Initializing extension manager.
May 05 01:30:09.539913 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: __init__.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.540002 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: __pycache__ {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.540062 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: address_group.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.540326 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.540522 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: address-group
May 05 01:30:09.540589 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: address_scope.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.540912 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.541129 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: address-scope
May 05 01:30:09.541251 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: admin_state_down_before_update.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.541553 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.541802 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
May 05 01:30:09.541930 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: agent.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.542238 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.542376 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: agent
May 05 01:30:09.542440 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: agent_resources_synced.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.542646 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.542780 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension agent-resources-synced not supported by any of loaded plugins
May 05 01:30:09.542848 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: allowedaddresspairs.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.543421 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.543570 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: allowed-address-pairs
May 05 01:30:09.543666 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: auto_allocated_topology.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.543872 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.544053 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: auto-allocated-topology
May 05 01:30:09.544178 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: availability_zone.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.544534 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.544794 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: availability_zone
May 05 01:30:09.544862 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: availability_zone_filter.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.545059 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.545172 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension availability_zone_filter not supported by any of loaded plugins
May 05 01:30:09.545307 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: data_plane_status.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.545531 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.545662 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension data-plane-status not supported by any of loaded plugins
May 05 01:30:09.545721 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: default_subnetpools.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.545990 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.546218 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: default-subnetpools
May 05 01:30:09.546338 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: dhcpagentscheduler.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.546780 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.547017 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: dhcp_agent_scheduler
May 05 01:30:09.547079 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: dns.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.547301 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.547441 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: dns-integration
May 05 01:30:09.547504 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: dns_domain_ports.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.547719 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.547866 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: dns-domain-ports
May 05 01:30:09.547963 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.548132 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.548258 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: dns-integration-domain-keywords
May 05 01:30:09.548334 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: dvr.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.548550 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.548683 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension dvr not supported by any of loaded plugins
May 05 01:30:09.548772 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: ecmp_routes.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.548963 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.549108 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension ecmp_routes not supported by any of loaded plugins
May 05 01:30:09.549168 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: empty_string_filtering.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.549381 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.549518 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: empty-string-filtering
May 05 01:30:09.549588 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.549771 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.549923 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
May 05 01:30:09.549995 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.550179 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.550319 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
May 05 01:30:09.550392 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: external_net.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.550573 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.550709 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: external-net
May 05 01:30:09.550797 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: extra_dhcp_opt.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.550980 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.551115 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: extra_dhcp_opt
May 05 01:30:09.551189 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: extraroute.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.551368 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.551508 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: extraroute
May 05 01:30:09.551575 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: extraroute_atomic.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.551816 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.551946 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: extraroute-atomic
May 05 01:30:09.552022 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: filter_validation.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.552221 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.552357 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: filter-validation
May 05 01:30:09.552411 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: fip_pf_description.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.552608 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.552736 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
May 05 01:30:09.552830 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: fip_pf_detail.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.553011 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.553145 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: floating-ip-port-forwarding-detail
May 05 01:30:09.553254 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: fip_pf_port_range.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.553439 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.553581 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
May 05 01:30:09.553656 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: fip_port_details.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.553852 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.554020 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: fip-port-details
May 05 01:30:09.554082 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: flavors.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.554412 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.554545 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: flavors
May 05 01:30:09.554601 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.554889 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.555029 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
May 05 01:30:09.555097 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: floatingip_pools.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.555324 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.555465 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: floatingip-pools
May 05 01:30:09.555534 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: ip_allocation.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.555736 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.555900 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension ip_allocation not supported by any of loaded plugins
May 05 01:30:09.555972 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: ip_substring_port_filtering.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.556178 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.556298 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension ip-substring-filtering not supported by any of loaded plugins
May 05 01:30:09.556375 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l2_adjacency.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.556552 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.556688 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension l2_adjacency not supported by any of loaded plugins
May 05 01:30:09.556777 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.557121 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.557319 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: router
May 05 01:30:09.557385 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_conntrack_helper.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.557886 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.558028 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension l3-conntrack-helper not supported by any of loaded plugins
May 05 01:30:09.558104 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.558296 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.558436 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: enable-default-route-bfd
May 05 01:30:09.558498 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.558692 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.558840 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: enable-default-route-ecmp
May 05 01:30:09.558909 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_ext_gw_mode.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.559087 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.559220 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: ext-gw-mode
May 05 01:30:09.559290 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_ext_ha_mode.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.559466 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.559598 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: l3-ha
May 05 01:30:09.559669 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.559865 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.559995 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
May 05 01:30:09.560063 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_extra_gws.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.560244 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.560379 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: external-gateway-multihoming
May 05 01:30:09.560441 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_flavors.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.560619 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.560752 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: l3-flavors
May 05 01:30:09.560840 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_ndp_proxy.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.561076 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.561225 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension l3-ndp-proxy not supported by any of loaded plugins
May 05 01:30:09.561287 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.561473 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.561601 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
May 05 01:30:09.561670 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: l3agentscheduler.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.562146 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.562275 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension l3_agent_scheduler not supported by any of loaded plugins
May 05 01:30:09.562346 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: local_ip.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.562603 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.562743 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension local_ip not supported by any of loaded plugins
May 05 01:30:09.562811 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: logging.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.563057 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.563193 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension logging not supported by any of loaded plugins
May 05 01:30:09.563263 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: metering.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.563621 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.563774 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension metering not supported by any of loaded plugins
May 05 01:30:09.563857 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.564068 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.564192 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
May 05 01:30:09.564257 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: multiprovidernet.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.564443 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.564572 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: multi-provider
May 05 01:30:09.564642 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: netmtu.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.564839 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.564973 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: net-mtu
May 05 01:30:09.565045 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: netmtu_writable.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.565266 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.565391 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: net-mtu-writable
May 05 01:30:09.565463 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: network_availability_zone.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.565664 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.565828 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: network_availability_zone
May 05 01:30:09.565902 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: network_ha.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.566071 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.566204 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension network_ha not supported by any of loaded plugins
May 05 01:30:09.566279 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: network_ip_availability.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.566466 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.566596 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: network-ip-availability
May 05 01:30:09.566661 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: network_ip_availability_details.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.566876 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.567009 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: network-ip-availability-details
May 05 01:30:09.567110 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: network_segment_range.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.567400 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.567534 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension network-segment-range not supported by any of loaded plugins
May 05 01:30:09.567596 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: pagination.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.567806 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.567870 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: pagination
May 05 01:30:09.567960 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_device_profile.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.568126 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.568265 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: port-device-profile
May 05 01:30:09.568333 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_hardware_offload_type.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.568502 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.568638 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-hardware-offload-type not supported by any of loaded plugins
May 05 01:30:09.568711 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.568903 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.569031 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
May 05 01:30:09.569116 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_hints.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.569307 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.569442 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-hints not supported by any of loaded plugins
May 05 01:30:09.569509 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_mac_address_override.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.569684 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.569837 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-mac-override not supported by any of loaded plugins
May 05 01:30:09.569894 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_mac_address_regenerate.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.570112 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.570256 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: port-mac-address-regenerate
May 05 01:30:09.570310 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_numa_affinity_policy.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.570622 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.570622 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: port-numa-affinity-policy
May 05 01:30:09.570697 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.570895 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.571026 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: port-numa-affinity-policy-socket
May 05 01:30:09.571073 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_resource_request.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.571263 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.571394 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-resource-request not supported by any of loaded plugins
May 05 01:30:09.571455 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_resource_request_groups.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.571651 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.571849 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-resource-request-groups not supported by any of loaded plugins
May 05 01:30:09.571849 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: port_trusted_vif.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.572031 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.572162 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-trusted-vif not supported by any of loaded plugins
May 05 01:30:09.572273 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: portbindings.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.572412 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.572712 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: binding
May 05 01:30:09.572712 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: portbindings_extended.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.572873 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.573007 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: binding-extended
May 05 01:30:09.573072 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: portsecurity.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.573286 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.573421 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: port-security
May 05 01:30:09.573484 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: project_id.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.573666 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.573741 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: project-id
May 05 01:30:09.573831 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: providernet.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.574011 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.574145 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: provider
May 05 01:30:09.574215 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qinq.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.574409 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.574544 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: qinq
May 05 01:30:09.574614 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.575335 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.575487 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos not supported by any of loaded plugins
May 05 01:30:09.575569 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_bw_limit_direction.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.575775 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.575947 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
May 05 01:30:09.575980 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.576173 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.576313 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
May 05 01:30:09.576378 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_default.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.576547 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.576680 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-default not supported by any of loaded plugins
May 05 01:30:09.576747 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_fip.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.576979 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.577065 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-fip not supported by any of loaded plugins
May 05 01:30:09.577135 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_fip_network_policy.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.577350 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.577475 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-fip-network-policy not supported by any of loaded plugins
May 05 01:30:09.577545 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_gateway_ip.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.577722 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.577924 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-gateway-ip not supported by any of loaded plugins
May 05 01:30:09.577998 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_port_network_policy.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.578184 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.578316 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-port-network-policy not supported by any of loaded plugins
May 05 01:30:09.578384 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.578573 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.578705 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-pps-minimum not supported by any of loaded plugins
May 05 01:30:09.578789 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.578980 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.579123 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
May 05 01:30:09.579173 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_pps_rule.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.579369 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.579504 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-pps not supported by any of loaded plugins
May 05 01:30:09.579571 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_rule_type_details.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.579745 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.579904 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-rule-type-details not supported by any of loaded plugins
May 05 01:30:09.579965 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_rule_type_filter.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.580174 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.580262 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-rule-type-filter not supported by any of loaded plugins
May 05 01:30:09.580332 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: qos_rules_alias.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.580518 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.580648 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension qos-rules-alias not supported by any of loaded plugins
May 05 01:30:09.580715 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: quota_check_limit.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.580912 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.581050 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: quota-check-limit
May 05 01:30:09.581102 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: quota_check_limit_default.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.581344 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.581465 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: quota-check-limit-default
May 05 01:30:09.581533 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: quotasv2.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.581962 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.582159 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: quotas
May 05 01:30:09.582233 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: quotasv2_detail.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.582505 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.582577 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: quota_details
May 05 01:30:09.582646 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: rbac.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.583107 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.583248 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: rbac-policies
May 05 01:30:09.583310 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: rbac_address_group.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.583520 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.583660 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension rbac-address-group not supported by any of loaded plugins
May 05 01:30:09.583732 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: rbac_address_scope.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.583934 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.584100 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: rbac-address-scope
May 05 01:30:09.584162 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: rbac_security_groups.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.584358 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.584496 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: rbac-security-groups
May 05 01:30:09.584578 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: rbac_subnetpool.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.584748 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.584905 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension rbac-subnetpool not supported by any of loaded plugins
May 05 01:30:09.584968 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: revisionifmatch.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.585144 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.585411 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: revision-if-match
May 05 01:30:09.585411 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: revisions.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.585727 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.586257 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: standard-attr-revisions
May 05 01:30:09.586348 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: router_availability_zone.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.586503 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.586704 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: router_availability_zone
May 05 01:30:09.586704 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: router_enable_snat.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.586937 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.587066 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: router-enable-snat
May 05 01:30:09.587066 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: routerservicetype.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.587263 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.587380 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension router-service-type not supported by any of loaded plugins
May 05 01:30:09.587461 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: security_groups_default_rules.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.587792 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.587943 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: security-groups-default-rules
May 05 01:30:09.588046 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.588230 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.588328 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: security-groups-normalized-cidr
May 05 01:30:09.588387 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: security_groups_port_filtering.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.588562 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.588681 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension port-security-groups-filtering not supported by any of loaded plugins
May 05 01:30:09.588799 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: security_groups_remote_address_group.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.588972 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.589086 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: security-groups-remote-address-group
May 05 01:30:09.589171 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.589424 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.589521 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: security-groups-rules-belongs-to-default-sg
May 05 01:30:09.589602 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: security_groups_shared_filtering.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.589736 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.589875 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: security-groups-shared-filtering
May 05 01:30:09.589966 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: securitygroup.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.591257 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.591395 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: security-group
May 05 01:30:09.591457 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: segment.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.591712 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.591867 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension segment not supported by any of loaded plugins
May 05 01:30:09.591933 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.592127 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.592252 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
May 05 01:30:09.592332 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: servicetype.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.592521 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.592662 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: service-type
May 05 01:30:09.592732 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: sorting.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.592946 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.593008 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: sorting
May 05 01:30:09.593078 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: standard_attr_segment.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.593268 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.593403 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension standard-attr-segment not supported by any of loaded plugins
May 05 01:30:09.593470 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: standardattrdescription.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.593669 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.593826 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: standard-attr-description
May 05 01:30:09.593879 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: stateful_security_group.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.594070 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.594203 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: stateful-security-group
May 05 01:30:09.594277 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: stdattrs_common.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.594437 np0000171612 neutron-ovn-maintenance-worker[104604]: WARNING neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
May 05 01:30:09.594506 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.594694 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.594876 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: subnet-dns-publish-fixed-ip
May 05 01:30:09.594951 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnet_external_network.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.595140 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.595277 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: subnet-external-network
May 05 01:30:09.595342 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnet_onboard.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.595544 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.595807 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension subnet_onboard not supported by any of loaded plugins
May 05 01:30:09.595888 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnet_segmentid_writable.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.596077 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.596215 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
May 05 01:30:09.596291 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnet_service_types.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.596478 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.596616 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: subnet-service-types
May 05 01:30:09.596687 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnetallocation.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.596902 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.597039 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: subnet_allocation
May 05 01:30:09.597113 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.597366 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.597510 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: subnetpool-prefix-ops
May 05 01:30:09.597585 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: tag_creation.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.597772 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.597924 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: tag-creation
May 05 01:30:09.597998 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.598183 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.598325 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
May 05 01:30:09.598417 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: tagging.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.599127 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.599245 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: standard-attr-tag
May 05 01:30:09.599328 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: timestamp.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.599531 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.599670 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: standard-attr-timestamp
May 05 01:30:09.599790 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: trunk.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.600007 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.600149 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension trunk not supported by any of loaded plugins
May 05 01:30:09.600218 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: trunk_details.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.600400 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.600537 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension trunk-details not supported by any of loaded plugins
May 05 01:30:09.600611 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: uplink_status_propagation.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.600826 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.600954 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension uplink-status-propagation not supported by any of loaded plugins
May 05 01:30:09.601028 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.601229 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.601383 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
May 05 01:30:09.601456 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loading extension file: vlantransparent.py {{(pid=104604) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 05 01:30:09.601653 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=104604) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 05 01:30:09.601812 np0000171612 neutron-ovn-maintenance-worker[104604]: INFO neutron.api.extensions [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Loaded extension: vlan-transparent
May 05 01:30:09.604065 np0000171612 neutron-ovn-maintenance-worker[104604]: WARNING oslo_service.backend._threading.service [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
May 05 01:30:09.604367 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Full set of CONF: {{(pid=104604) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
May 05 01:30:09.604425 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ******************************************************************************** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 05 01:30:09.604499 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Configuration options gathered from: {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 05 01:30:09.604570 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 05 01:30:09.604674 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 05 01:30:09.604734 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ================================================================================ {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 05 01:30:09.604875 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] agent_down_time                = 75 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.604960 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] allow_automatic_dhcp_failover  = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605049 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] allow_bulk                     = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605277 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605368 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] api_extensions_path            =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605462 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] api_paste_config               = api-paste.ini {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605560 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] api_workers                    = 4 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605651 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] auth_strategy                  = keystone {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605743 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605870 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] bind_host                      = 0.0.0.0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.605964 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] bind_port                      = 9696 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606052 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] client_socket_timeout          = 900 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606146 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] config_dir                     = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606214 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606304 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] config_source                  = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606398 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] control_exchange               = neutron {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606497 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] core_plugin                    = ml2 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606578 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] debug                          = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606666 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] default_availability_zones     = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606829 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.606923 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] dhcp_agent_notification        = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607012 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] dhcp_agents_per_network        = 1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607097 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] dhcp_lease_duration            = 86400 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607191 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] dhcp_load_type                 = networks {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607296 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] dns_domain                     = openstacklocal {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607414 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607510 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_default_route_bfd       = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607598 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_default_route_ecmp      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607683 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_dvr                     = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607773 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_new_agents              = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607873 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_services_on_agents_with_admin_state_down = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.607959 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_snat_by_default         = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608043 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] enable_traditional_dhcp        = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608127 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] external_dns_driver            = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608210 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] filter_validation              = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608303 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] global_physnet_mtu             = 1430 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608369 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] graceful_shutdown_timeout      = 60 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608492 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] host                           = np0000171612 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608581 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] host_dvr_for_dhcp              = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608668 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] http_retries                   = 3 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608767 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] instance_format                = [instance: %(uuid)s]  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608869 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.608954 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ipam_driver                    = internal {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609037 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ipv6_pd_enabled                = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609122 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] l3_ha                          = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609250 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609337 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] l3_ha_network_physical_name    =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609427 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] l3_ha_network_type             =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609517 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_color                      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609607 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_config_append              = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609697 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609805 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_dir                        = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609902 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_file                       = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.609953 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_options                    = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610045 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_rotate_interval            = 1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610134 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_rotate_interval_type       = days {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610229 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] log_rotation_type              = none {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610288 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610354 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610443 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610533 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610599 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610704 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_allowed_address_pair       = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610827 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_dns_nameservers            = 5 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.610917 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_header_line                = 16384 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611003 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_l3_agents_per_router       = 3 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611090 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_logfile_count              = 30 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611178 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_logfile_size_mb            = 200 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611264 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_routes                     = 30 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611349 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] max_subnet_host_routes         = 20 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611439 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] my_ip                          = 199.204.45.78 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611528 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe4d:79ff {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611615 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] network_auto_schedule          = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611698 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] network_link_prefix            = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611802 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611894 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] notify_nova_on_port_data_changes = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.611982 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] notify_nova_on_port_status_changes = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612072 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] pagination_max_limit           = -1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612157 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] periodic_fuzzy_delay           = 5 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612281 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] periodic_interval              = 40 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612359 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] publish_errors                 = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612420 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rate_limit_burst               = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612585 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rate_limit_except_level        = CRITICAL {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612585 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rate_limit_interval            = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612670 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] router_distributed             = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612745 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rpc_resources_processing_step  = 20 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612861 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rpc_response_max_timeout       = 600 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.612928 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rpc_response_timeout           = 60 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613026 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rpc_state_report_workers       = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613106 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] rpc_workers                    = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613195 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] send_events_interval           = 2 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613304 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] service_plugins                = ['ovn-router'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613400 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] setproctitle                   = on {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613479 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] shell_completion               = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613614 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] state_path                     = /opt/stack/data/neutron {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613808 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] syslog_log_facility            = LOG_USER {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613856 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] tcp_keepidle                   = 600 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.613929 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] transport_url                  = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614015 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] use_journal                    = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614100 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] use_json                       = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614189 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] use_stderr                     = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614303 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] use_syslog                     = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614378 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] vlan_qinq                      = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614437 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] vlan_transparent               = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614524 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] wsgi_default_pool_size         = 100 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614645 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] wsgi_keep_alive                = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614686 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614778 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] wsgi_server_debug              = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.614885 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_concurrency.disable_process_locking = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.614982 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615084 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.connection_string     = messaging:// {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615176 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.enabled               = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615271 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.es_doc_type           = notification {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615367 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.es_scroll_size        = 10000 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615463 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.es_scroll_time        = 2m {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615536 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.filter_error_trace    = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615627 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.hmac_keys             = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615725 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.sentinel_service_name = mymaster {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615873 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.socket_timeout        = 0.1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615944 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.trace_requests        = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.615999 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler.trace_sqlalchemy      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.616126 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler_jaeger.process_tags   = {} {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.616197 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler_jaeger.service_name_prefix = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.616530 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] profiler_otlp.service_name_prefix = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.616597 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.616636 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.616739 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617081 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617081 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617081 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.enforce_new_defaults = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617273 np0000171612 neutron-ovn-maintenance-worker[104604]: WARNING oslo_config.cfg [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
May 05 01:30:09.617328 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.enforce_scope      = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617425 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.policy_default_rule = default {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617526 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617596 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617702 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617814 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617885 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.617983 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618064 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618153 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_policy.remote_timeout     = 60.0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618247 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618345 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] service_providers.service_provider = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618435 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.auth_section              = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618534 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.auth_type                 = password {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618621 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.auth_url                  = https://199.204.45.78/identity {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618707 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.cafile                    = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618822 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.certfile                  = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618909 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.collect_timing            = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.618995 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.default_domain_id         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619084 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.default_domain_name       = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619175 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.domain_id                 = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619256 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.domain_name               = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619347 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.endpoint_type             = public {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619436 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.insecure                  = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619520 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.keyfile                   = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619611 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.password                  = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619698 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.project_domain_id         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619804 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.project_domain_name       = Default {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619900 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.project_id                = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.619992 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.project_name              = service {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620082 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.region_name               = RegionOne {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620640 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.split_loggers             = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620640 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.system_scope              = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620640 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.timeout                   = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620640 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.trust_id                  = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620640 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.user_domain_id            = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620832 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.user_domain_name          = Default {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620832 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.user_id                   = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620904 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] nova.username                  = nova {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.620997 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.auth_section         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621088 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.auth_type            = password {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621176 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.cafile               = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621287 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.certfile             = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621414 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.collect_timing       = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621459 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.endpoint_type        = public {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621557 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.insecure             = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621649 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.keyfile              = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621728 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.region_name          = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621833 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.split_loggers        = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.621921 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] placement.timeout              = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622009 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.auth_section            = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622097 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.auth_type               = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622185 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.cafile                  = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622268 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.certfile                = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622366 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.collect_timing          = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622479 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.connect_retries         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622583 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.connect_retry_delay     = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622672 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.enable_notifications    = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622769 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.endpoint_override       = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622865 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.insecure                = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.622952 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.interface               = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623056 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.keyfile                 = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623121 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.max_version             = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623211 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.min_version             = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623299 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.region_name             = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623386 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.retriable_status_codes  = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623470 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.service_name            = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623554 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.service_type            = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623639 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.split_loggers           = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623722 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.status_code_retries     = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623825 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.status_code_retry_delay = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623910 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.timeout                 = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.623995 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.valid_interfaces        = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624079 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ironic.version                 = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624171 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] cli_script.dry_run             = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624273 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624372 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624456 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624549 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624635 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624723 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.624827 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625040 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625040 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625099 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.hostname = np0000171612 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625166 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625275 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625363 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625449 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625536 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625626 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625742 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625860 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625890 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.625977 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626070 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626141 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626229 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626315 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626405 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626485 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626570 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626680 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626726 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626828 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.626913 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.ssl      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627052 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627081 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627162 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627248 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627335 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.ssl_version =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627419 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627507 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_notifications.driver = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627594 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_notifications.retry = -1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627687 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627783 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_messaging_notifications.transport_url = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627872 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_reports.file_event_handler = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.627976 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_reports.file_event_handler_interval = 1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628038 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_reports.log_dir           = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628124 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628211 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.default_quota           = -1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628300 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628419 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_floatingip        = 50 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628533 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_network           = 100 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628625 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_port              = 500 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628714 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628821 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_router            = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628908 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_security_group    = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.628994 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_security_group_rule = 100 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629080 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.quota_subnet            = 100 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629166 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] QUOTAS.track_quota_usage       = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629294 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629381 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.group                  = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629466 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.helper_command         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629553 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.log_daemon_traceback   = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629643 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629731 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.thread_pool_size       = 16 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629834 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep.user                   = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.629928 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630014 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.group     = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630099 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.helper_command = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630185 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630278 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630359 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630458 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_dhcp_release.user      = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630551 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630622 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.group        = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630707 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.helper_command = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630838 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.630900 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631000 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631072 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_ovs_vsctl.user         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631163 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.capabilities = [21] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631248 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.group        = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631333 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.helper_command = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631421 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.log_daemon_traceback = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631509 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631595 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.thread_pool_size = 16 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631681 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_namespace.user         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631784 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.capabilities = [12] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631870 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.group        = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.631980 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.helper_command = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632029 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.log_daemon_traceback = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632123 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632208 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.thread_pool_size = 16 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632294 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_conntrack.user         = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632395 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.capabilities      = [12, 21] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632471 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.group             = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632553 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.helper_command    = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632639 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.log_daemon_traceback = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632728 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632829 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.thread_pool_size  = 16 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.632917 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] privsep_link.user              = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633020 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633105 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633193 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] AGENT.use_helper_for_ns_read   = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633301 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] SECURITYGROUP.enable_ipset     = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633396 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] SECURITYGROUP.enable_security_group = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633481 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] SECURITYGROUP.firewall_driver  = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633587 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633664 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] experimental.ipv6_pd_enabled   = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633774 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633874 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.external_network_type      = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.633971 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634066 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.overlay_ip_version         = 4 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634153 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.path_mtu                   = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634242 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.physical_network_mtus      = {} {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634340 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.project_network_types      = ['geneve'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634432 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634530 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634630 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634728 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2_type_geneve.max_header_size = 38 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634838 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.634936 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635030 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] network_log.burst_limit        = 25 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635116 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] network_log.local_output_log_base = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635204 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] network_log.rate_limit         = 100 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635299 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.asyncio_connection    = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635384 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.asyncio_slave_connection = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635475 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.backend               = sqlalchemy {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635566 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.connection            = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635655 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.connection_debug      = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635745 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.connection_parameters =  {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635851 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.connection_recycle_time = 3600 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.635939 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.connection_trace      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636027 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.db_inc_retry_interval = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636114 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.db_max_retries        = 20 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636203 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.db_max_retry_interval = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636289 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.db_retry_interval     = 1 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636375 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.max_overflow          = 50 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636461 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.max_pool_size         = 5 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636547 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.max_retries           = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636638 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636724 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.mysql_wsrep_sync_wait = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636828 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.pool_timeout          = None {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.636917 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.retry_interval        = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637005 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.slave_connection      = **** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637090 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.sqlite_synchronous    = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637179 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.synchronous_reader    = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637326 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] database.use_db_reconnect      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637422 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.bfd_min_rx                 = 1000 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637509 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.bfd_min_tx                 = 100 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637949 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.bfd_mult                   = 3 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637949 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.broadcast_arps_to_all_routers = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637949 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637949 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.637949 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.dns_records_ovn_owned      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638144 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.dns_servers                = [] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638144 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.enable_distributed_floating_ip = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638199 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.fdb_age_threshold          = 300 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638271 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ha_failover_strategy       = normal {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638363 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.live_migration_activation_strategy = rarp {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638451 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.localnet_learn_fdb         = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638558 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.mac_binding_age_threshold  = 300 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638665 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.neutron_sync_mode          = log {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638797 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638857 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.638950 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639043 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_metadata_enabled       = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639134 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639224 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639321 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_nb_connection          = ['ssl:199.204.45.78:6641'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639411 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639498 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_router_indirect_snat   = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639589 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639677 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639793 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_sb_connection          = ['ssl:199.204.45.78:6642'] {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639888 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.639975 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovs_create_tap             = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640063 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640156 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovsdb_log_level            = INFO {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640249 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640338 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640423 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.stateless_nat_enabled      = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640512 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640605 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVS.bridge_mac_table_size      = 50000 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640692 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVS.igmp_flood                 = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.640796 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVS.igmp_flood_reports         = True {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641052 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVS.igmp_flood_unregistered    = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641052 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVS.igmp_snooping_enable       = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641052 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] OVS.ovsdb_timeout              = 10 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641148 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641249 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641382 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.641493 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG cotyledon.oslo_config_glue [None req-cf0bf223-a096-42b5-b532-92698118373a None None] ******************************************************************************** {{(pid=104604) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 05 01:30:09.641493 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG oslo_service.backend._threading.service [None req-cf0bf223-a096-42b5-b532-92698118373a None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x7c4992872c60> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x7c4992872c60> {{(pid=104604) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
May 05 01:30:09.670236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=105368) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
May 05 01:30:09.670681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 05 01:30:09.670681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 05 01:30:09.670681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 05 01:30:09.670681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 05 01:30:09.670681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 05 01:30:09.670881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.670942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.671806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.672906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.673999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.674878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000171612 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.675952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.676942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.677947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.204.45.78 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe4d:79ff {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678793 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.678976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.679925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.680923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 05 01:30:09.681701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.681852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.681982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682771 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.682976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.683929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.204.45.78/identity {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.684994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.685959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687771 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.687988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.688722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.693826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.693940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000171612 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694793 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.694964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695854 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.695952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.696944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.697964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698772 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.698998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699602 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.699998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700767 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.700959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.701992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.702980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.204.45.78:6641'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.703991 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.204.45.78:6642'] {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704772 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.704972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.705055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.705098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.705530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.705629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.705672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 05 01:30:09.705736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105368) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 05 01:30:09.706017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [105368] {{(pid=105368) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
May 05 01:30:09.707272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-166748', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-27901'] for process (None), after_init {{(pid=105368) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 05 01:30:09.708419 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
May 05 01:30:09.756042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 05 01:30:09.756163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 05 01:30:09.756163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 05 01:30:09.756336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.756837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.758752 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.78:6641: connecting...
May 05 01:30:09.766021 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.78:6641: connected
May 05 01:30:09.793710 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
May 05 01:30:09.825377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 05 01:30:09.825377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 05 01:30:09.825583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.825583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=105368) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 05 01:30:09.827687 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.78:6642: connecting...
May 05 01:30:09.833880 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.78:6642: connected
May 05 01:30:09.873862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c498c9e6c30>> floatingip precommit_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c498c9e6c30>> floatingip precommit_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c498c9e6c30>> floatingip before_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c498c9e6c30>> floatingip before_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c498c9e6c30>> port after_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c498c9e6c30>> port precommit_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-11280'] for port_forwarding_plugin (None), after_init {{(pid=105368) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 05 01:30:09.874590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c498d19a120>> port_forwarding after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c498d19a120>> router_interface after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c498d19a120>> port_forwarding after_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c498d19a120>> port_forwarding after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.874953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c498d19a120>> router_interface after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.875126 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 05 01:30:09.875204 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
May 05 01:30:09.875692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x7c498c9e5100>> segment before_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.875812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x7c498c9e5100>> subnet precommit_delete_associations 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> segment_host_mapping after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> segment_host_mapping after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> port after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> port after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> port after_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> segment after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> subnet after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> subnet after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.876687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c498c9e6cc0>> subnet after_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.942313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c498c9ba1b0>> subnet after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.942313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c498c9ba1b0>> subnet after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.942313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c498c9ba1b0>> subnet before_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.942664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c498c9ba1b0>> subnet before_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7c498c9ec470>> security_group after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.945139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7c498c9ec470>> port after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.945181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-791035'] for logging-plugin (None), after_init {{(pid=105368) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 05 01:30:09.945372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c498c9ba1e0>> security_group_rule after_create 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.945430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c498c9ba1e0>> security_group_rule after_update 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:09.945518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c498c9ba1e0>> security_group_rule after_delete 55550000, False {{(pid=105368) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 05 01:30:19.399887 np0000171612 neutron-ovn-maintenance-worker[104604]: DEBUG dbcounter [-] [104604] Writing DB stats neutron:SELECT=4 {{(pid=104604) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:30:19.946739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-05-05 01:30:19.946314 {{(pid=105368) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
May 05 01:30:19.949444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync port groups started @ 2026-05-05 01:30:19.948971 {{(pid=105368) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
May 05 01:30:19.967399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writer thread running {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 05 01:30:19.980416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync port groups completed @ 2026-05-05 01:30:19.980063 {{(pid=105368) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
May 05 01:30:19.980416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-05-05 01:30:19.980292 {{(pid=105368) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}}
May 05 01:30:20.278721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.services.segments.db [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x7c498d65ca70>,) {'filters': {'network_id': ['c4593571-c367-4d2b-9f28-1c78cd90fd82'], 'is_dynamic': False}} {{(pid=105368) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 05 01:30:20.284472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync metadata ports started {{(pid=105368) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}}
May 05 01:30:20.346438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync metadata ports completed {{(pid=105368) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}}
May 05 01:30:20.346574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=105368) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}}
May 05 01:30:20.405822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-05-05 01:30:20.405581 {{(pid=105368) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}}
May 05 01:30:20.406475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 05 01:30:20.406649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=105368) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}}
May 05 01:30:20.406715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=105368) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}}
May 05 01:30:20.406840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-05-05 01:30:20.406718 {{(pid=105368) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}}
May 05 01:30:20.406940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync ACLs started @ 2026-05-05 01:30:20.406846 {{(pid=105368) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
May 05 01:30:20.442823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync ACLs completed @ 2026-05-05 01:30:20.442313 {{(pid=105368) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}}
May 05 01:30:20.443006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync routers and router ports started @ 2026-05-05 01:30:20.442832 {{(pid=105368) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}}
May 05 01:30:20.502858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 05 01:30:20.502999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync routers and router ports completed @ 2026-05-05 01:30:20.502734 {{(pid=105368) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}}
May 05 01:30:20.503123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync port QoS policies started @ 2026-05-05 01:30:20.502979 {{(pid=105368) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}}
May 05 01:30:20.503199 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] Starting OVNClientQosExtension
May 05 01:30:20.519399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync port QoS policies completed @ 2026-05-05 01:30:20.519138 {{(pid=105368) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}}
May 05 01:30:20.519580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync Floating IP QoS policies started @ 2026-05-05 01:30:20.519399 {{(pid=105368) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}}
May 05 01:30:20.519664 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] Starting OVNClientQosExtension
May 05 01:30:20.542226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-05-05 01:30:20.542013 {{(pid=105368) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}}
May 05 01:30:20.542363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync Floating IP NAT rules started @ 2026-05-05 01:30:20.542208 {{(pid=105368) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}}
May 05 01:30:20.542605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-05-05 01:30:20.542481 {{(pid=105368) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}}
May 05 01:30:20.542719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-fa8fada0-ac33-4175-a051-05c43d578f3f None None] OVN-Northbound DB sync process completed @ 2026-05-05 01:30:20.542592 {{(pid=105368) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
May 05 01:30:29.949041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-05-05 01:30:29.948439 {{(pid=105368) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}}
May 05 01:30:29.950073 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 05 01:30:29.951444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] OVN-SB Sync hostname and physical networks started @ 2026-05-05 01:30:29.950957 {{(pid=105368) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}}
May 05 01:30:29.956924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.956924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.956924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.956924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.957997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.958971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.961916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.961916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=105368) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 05 01:30:29.961916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.961916 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
May 05 01:30:29.962672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] Agents populated in the agent cache: {{(pid=105368) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:310}}
May 05 01:30:29.963252 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
May 05 01:30:29.964236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.965296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] - Agent type: OVN Controller Gateway agent, hostname: np0000171612, name: db4f2cd9-f03d-427b-86b1-bf324a42ead8 {{(pid=105368) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 05 01:30:29.967570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.967993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.968085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.968179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.971366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] - Agent type: OVN Neutron agent, hostname: np0000171612, name: db4f2cd9-f03d-427b-86b1-bf324a42ead8 {{(pid=105368) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 05 01:30:29.972801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.972878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.972968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.973227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.973298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.973359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.973430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.974338 np0000171612 neutron-ovn-maintenance-worker[105368]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] New host np0000171612 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
May 05 01:30:29.974486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.974552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.974859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.974934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.974989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.975493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=105368) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 05 01:30:29.984454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:29.984867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:29.984867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.018222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG oslo_concurrency.lockutils [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=105368) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 05 01:30:30.018583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG oslo_concurrency.lockutils [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=105368) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 05 01:30:30.062083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron_lib.callbacks.manager [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-220787', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-1490992'] for segment_host_mapping (None), after_create {{(pid=105368) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 05 01:30:30.083801 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.098 seconds
May 05 01:30:30.083801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.084162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.084162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.090427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.services.segments.db [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] Segments {'fd02efee-ef31-4e95-abc3-f00ac4a17a72'} mapped to the host np0000171612 {{(pid=105368) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
May 05 01:30:30.100856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG oslo_concurrency.lockutils [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.082s {{(pid=105368) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 05 01:30:30.101099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] OVN-SB Sync hostname and physical networks completed @ 2026-05-05 01:30:30.100925 {{(pid=105368) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}}
May 05 01:30:30.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_dhcp_options_consistency finished in 0.060 seconds
May 05 01:30:30.145298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.145474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.145601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.146015 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
May 05 01:30:30.146158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.146286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.146396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.149392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:30.244317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.249974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.264282 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.118 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.268752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.269840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:30.270388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.270388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:30.272161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:30.272947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.001 seconds
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.274987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.275338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.336147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: No inconsistencies found. Skipping {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}}
May 05 01:30:30.336335 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.062 seconds
May 05 01:30:30.336608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.336731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.337027 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
May 05 01:30:30.342711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.346989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.347221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.350743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 05 01:30:30.353069 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.005 seconds
May 05 01:30:30.356910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.357100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.357560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.424114 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.066 seconds
May 05 01:30:30.424403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.424495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.424693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.531469 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_provider_distributed_ports finished in 0.107 seconds
May 05 01:30:30.531684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.531831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.533181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.596135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 05 01:30:30.597944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-8e00e236-466a-4b7a-8a3d-7f6c167f23c2 None None] OVN-Southbound DB sync process completed @ 2026-05-05 01:30:30.596969 {{(pid=105368) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}}
May 05 01:30:30.618518 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.086 seconds
May 05 01:30:30.619927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.619927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.619927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.620114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.620259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.620862 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.621772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.622116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:30.622439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.622439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:30.622889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:30.623593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.625494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:30.625494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.625494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:30.625494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.625494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.628547 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.db.ovn_hash_ring_db [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Cleaned up Hash Ring nodes older than 5 days
May 05 01:30:30.628737 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.005 seconds
May 05 01:30:30.629043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: configure_nb_global {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.629192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.629534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:30.631559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.632185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.633141 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:30.634251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.634802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task configure_nb_global failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.635635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:30.636244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:30.637016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:30.637481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 05 01:30:30.638120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.638525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.638525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.638525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:30:30.638923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.639227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.640006 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:30.640813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.641399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.641936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:30.642339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:30.642714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.642714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:30.642714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.642714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:30.643249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.643895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.644523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.651654 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_network_type_and_physnet finished in 0.008 seconds
May 05 01:30:30.651804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.652045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.652157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.652500 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds
May 05 01:30:30.652649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.652897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.653003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.653263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:30.653995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.654264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.654940 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:30.655903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.657075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:30.657306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:30.657555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.657555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:30.657555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.657555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.658500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:30.658866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:30.659232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.659232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:30.659232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.659232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:30.659232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.659232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.659881 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.001 seconds
May 05 01:30:30.660014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.660199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.660320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.660530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:30.661545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.661612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.661785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.662614 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.663907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.664346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:30.664643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.664643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:30.667304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:30.667832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.668468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:30.668468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.668468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:30.668468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.668468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.672998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=beffa84f-427a-4b23-b4cb-b577e6f22c52, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562', 'neutron:subnet_ids': 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:8b:e8:5a', name='lrp-a10e3648-9330-4d0c-8bd9-607a76e0fc73', networks=['fda3:6386:b65f::1/64'], options={}, peer=[], status={}) old= {{(pid=105368) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 05 01:30:30.674544 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 05 01:30:30.695929 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_router_distributed_flag finished in 0.028 seconds
May 05 01:30:30.696372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.696892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_router_static_routes {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.697524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.702404 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_router_static_routes finished in 0.005 seconds
May 05 01:30:30.702532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.703226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:30.703226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:30.727416 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_security_group_with_address_group finished in 0.024 seconds
May 05 01:30:30.727582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.728686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105368) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 05 01:30:30.746706 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82 synchronized; highest priority chassis db4f2cd9-f03d-427b-86b1-bf324a42ead8
May 05 01:30:30.747176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'neutron:router_id': '68bc1d51-8ada-4507-8adc-68e5fc8c2562'}),), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.747432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, chassis=db4f2cd9-f03d-427b-86b1-bf324a42ead8, priority=1, columns={}) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:30.747615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.748149 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event Traceback (most recent call last):
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2415, in link_network_ha_chassis_group
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self.create_transaction(
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.result = self.commit()
May 05 01:30:30.749933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise result.ex
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:30.750692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event 
May 05 01:30:32.754977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=3335bd45-3b43-4cab-adee-6878542e7572, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}, gateway_chassis=[], ha_chassis_group=d8637cb3-c087-4e8c-a12f-9ed394e0b660, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:91:eb:5a', name='lrp-7030fda3-53e6-4434-89f4-e189344bc2a9', networks=['172.24.5.41/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105368) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 05 01:30:32.756406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=7fd9e0d6-3489-4787-95b9-52098e0d5ea3, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, gateway_chassis=[], ha_chassis_group=d8637cb3-c087-4e8c-a12f-9ed394e0b660, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:91:eb:5a', name='lrp-7030fda3-53e6-4434-89f4-e189344bc2a9', networks=['172.24.5.41/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': 'db4f2cd9-f03d-427b-86b1-bf324a42ead8'}) old= {{(pid=105368) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 05 01:30:32.798700 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82 synchronized; highest priority chassis db4f2cd9-f03d-427b-86b1-bf324a42ead8
May 05 01:30:32.799461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'neutron:router_id': '68bc1d51-8ada-4507-8adc-68e5fc8c2562'}),), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:32.800024 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, chassis=db4f2cd9-f03d-427b-86b1-bf324a42ead8, priority=1, columns={}) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:32.800394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:32.801242 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event Traceback (most recent call last):
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2415, in link_network_ha_chassis_group
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:32.802123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self.create_transaction(
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.result = self.commit()
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise result.ex
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:32.802804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event 
May 05 01:30:33.022800 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82 synchronized; highest priority chassis db4f2cd9-f03d-427b-86b1-bf324a42ead8
May 05 01:30:33.023518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'neutron:router_id': '68bc1d51-8ada-4507-8adc-68e5fc8c2562'}),), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, chassis=db4f2cd9-f03d-427b-86b1-bf324a42ead8, priority=1, columns={}) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event Traceback (most recent call last):
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
May 05 01:30:33.026143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2415, in link_network_ha_chassis_group
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self.create_transaction(
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.result = self.commit()
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise result.ex
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 05 01:30:33.026680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 05 01:30:33.027188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:33.027188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 05 01:30:33.027188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:33.027188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event 
May 05 01:30:35.147319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:35.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:35.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:35.148722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:35.182196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:35.182707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.183605 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.184538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.184945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:35.185580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:35.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.187755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:35.187755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:35.187755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.187755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:35.187755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.187755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:35.622296 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.623333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:35.624092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:35.624977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:35.626003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.626651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:35.638205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:35.638581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:35.638713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:35.638856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:30:35.639478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:35.639889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.640987 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:35.642701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.643142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.644610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:35.644977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:35.645388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.645388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:35.645388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.645388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.647252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.647985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:35.653017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:35.653642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:35.653854 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:35.654088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:35.654642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:35.655101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.655941 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:35.656744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.657167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.657781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:35.658076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:35.658395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.658395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:35.658395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.658395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.658952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:35.659390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:35.659854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.659854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:35.659854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.659854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:35.660221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:35.660421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:35.660545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:35.660709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:35.661085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:35.661415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:35.661547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.662121 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.663044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.663363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:35.663701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.663701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:35.663896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:35.664311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:35.664735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:35.664735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:35.664735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:40.147951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:40.148722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:40.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:40.151575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:40.187863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:40.188450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.189461 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.191909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:40.192528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:40.193225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:40.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:40.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:40.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:40.619988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:40.620783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:40.621166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:40.622104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:40.622501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.623489 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.625539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.626264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:40.626988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.626988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:40.627440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:40.628305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.629318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:40.629318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.629318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:40.638700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:40.639103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:40.639171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:40.639327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:30:40.639869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:40.640207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.641052 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:40.642466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.643252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.644028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:40.644591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:40.645425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.645425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:40.645425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.645425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:40.645721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.646483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.647459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:40.653637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:40.653986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:40.654146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:40.654238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:40.654711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:40.655113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.656173 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:40.657058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.657924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.658568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:40.659191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:40.659741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.659741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:40.659741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.659741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.660311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:40.660723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:40.661373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.661373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:40.661373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.661373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:40.661373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:40.664036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:40.664159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:40.664327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:40.664641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:40.664898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:40.665013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.665621 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.666556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.667135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:40.667636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:40.668230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:40.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:40.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:40.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:40.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:40.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:40.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:45.148628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:45.149115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:45.149115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:45.149829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:45.183587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:45.184050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.185025 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.185818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:45.186497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.186497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:45.186918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:45.187814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.188604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:45.188604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.188604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:45.621048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:45.621520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:45.621832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:45.622828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:45.623400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.624801 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.628126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.628725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:45.629087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.629087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:45.630905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:45.631795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.632293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:45.632293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.632293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:45.639263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:45.639656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:45.639966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:45.640208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:30:45.641073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:45.641594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.642807 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:45.644319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.645002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.645919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:45.646468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:45.647138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.647138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:45.647138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.647138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:45.647379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.648191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.650522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:45.654115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:45.654402 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:45.654601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:45.654845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:45.655330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:45.655589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.656402 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:45.657838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.658644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.659411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:45.659782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:45.660205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.660205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:45.660205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.660205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.661253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:45.661832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:45.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:45.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:45.664210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:45.664640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:45.664939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:45.665319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:45.665713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:45.666148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:45.666369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.667056 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.668713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.669067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:45.669433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.669433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:45.670588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:45.671097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:45.671815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:45.671815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:45.671815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:50.149419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:50.152781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:50.152781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:50.152781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:50.183310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:50.183854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.184750 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.185740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.186343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:50.186927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:50.187659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.188559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:50.188559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:50.188559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.188559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:50.188559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.188559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:50.621873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:50.622227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:50.622325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:50.623584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:50.623995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.624935 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.626278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.626710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:50.627146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.627146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:50.627437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:50.627979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.628587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:50.628587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.628587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:50.639636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:50.640032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:50.640267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:50.640972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:30:50.641355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:50.641813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.642863 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:50.644272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.645171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.645651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:50.646404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:50.647142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.647142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:50.647142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.647142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:50.647382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.648354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.649035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:50.654380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:50.654605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:50.654701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:50.654871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:50.655289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:50.655547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.656151 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:50.657312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.658174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.659048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:50.659622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:50.660731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.660731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:50.660731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.660731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.661431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:50.662367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:50.663419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.663419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:50.663419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.663419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:50.664486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:50.664713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:50.664873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:50.665061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:50.665643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:50.666000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:50.666199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.667016 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.668108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.668530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:50.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.668941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:50.669403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:50.669912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:50.670467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:50.670467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:50.670467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:55.162042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:55.162042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:55.162042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:55.162042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:55.181802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:55.182232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.183206 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.184173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.184847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:55.185436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:55.186389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.187414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:55.187414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:55.187414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.187414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:55.187414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.187414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:55.622280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:55.623046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:55.623281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:55.624119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:55.624449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.628111 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.629541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.630281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:55.630947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.630947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:55.633158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:55.633737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.634622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:55.634622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.634622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:55.640231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:55.640591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:55.640792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:55.640853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:30:55.641320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:55.641637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.642398 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:55.643259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.643854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.644822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:55.645470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:55.645853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.647249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.648029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:55.654757 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:55.655060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:55.655178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:55.655300 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:55.655712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:55.656119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.656982 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:30:55.657613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.658119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.658880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:55.659426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:55.660020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.660020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:55.660020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.660020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.660630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:55.661410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:55.662285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.662285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:55.662285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.662285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:30:55.664814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:30:55.665188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:30:55.665370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:30:55.665601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:30:55.666104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:55.666494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:30:55.666682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.667346 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.668149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.668542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:30:55.668928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.668928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:30:55.669200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:30:55.669724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:30:55.670296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:30:55.670296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:30:55.670296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:00.151214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:00.152039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:00.152039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:00.153454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:00.189100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:00.189503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.190428 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.191221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.191827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:00.192462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:00.193332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.194294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:00.194294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:00.194294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.194294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:00.194294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.194294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:00.623028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:00.623439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:00.623572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:00.624803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:00.625234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.626442 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.627567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.628543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:00.629360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:00.630158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.631117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:00.631117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:00.631117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.631117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:00.631117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.631117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:00.640698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:00.641008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:00.641199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:00.641448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:00.642658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:00.643068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.644072 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:00.645418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.646287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.646981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:00.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:00.648285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.648285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:00.648285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.648285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:00.648654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.649506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.650326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:00.655201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:00.655498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:00.655660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:00.655868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:00.656356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:00.656665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.657505 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:00.658539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.659097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.659825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:00.660404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:00.661101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.661101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:00.661101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.661101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.662797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:00.663708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:00.664632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.664632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:00.664632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.664632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:00.665481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:00.665651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:00.665724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:00.665887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:00.666197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:00.666457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:00.666577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.667080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.667639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.668093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:00.668465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:00.668897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:00.669437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:00.669437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:00.669437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:00.669437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:00.669437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:00.669437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:05.151875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:05.152636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:05.152636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:05.153822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:05.185544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.188462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.189307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:05.189956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.190530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.191300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:05.623666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:05.624077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:05.624223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:05.625005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:05.625274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.626052 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.627171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.627676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:05.628138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.628138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:05.629507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:05.630040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.630610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:05.630610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.630610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:05.641037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:05.641353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:05.641475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:05.641613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:05.642363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:05.642923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.644993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:05.645505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.645989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.646289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:05.646563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:05.646925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.647318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:05.655446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:05.655704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:05.656026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:05.656026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:05.656524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:05.656938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.657715 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:05.659204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:05.659558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:05.659887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.660169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.660724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:05.661139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:05.661532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.661532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:05.661532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.661532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:05.665773 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:05.666011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:05.666096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:05.666210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:05.666635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:05.666971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:05.667141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.667814 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.668417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.668675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:05.668956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.668956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:05.669167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:05.669500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:05.669891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:05.669891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:05.669891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:10.152333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:10.153127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:10.153127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:10.154538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:10.183283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:10.183481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.184099 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.185045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.185684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:10.186279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:10.186976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.187828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:10.187828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:10.187828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.187828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:10.187828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.187828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:10.624306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:10.624711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:10.624932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:10.625722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:10.626057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.626906 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.627993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.628491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:10.628978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.628978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:10.629080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:10.629671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.630376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:10.630376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.630376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:10.641389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:10.641839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:10.642029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:10.642272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:10.642852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:10.643188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.644011 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:10.645045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.645884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.647053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:10.648074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:10.648718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.649528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.650485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:10.655866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:10.656179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:10.656362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:10.656571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:10.657037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:10.657322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.657965 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:10.659196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.659722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.660946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:10.661483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:10.662115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.662115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:10.662115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.662115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.663087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:10.664047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:10.664927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.664927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:10.664927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.664927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:10.666376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:10.666794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:10.667029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:10.667278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:10.667850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:10.668219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:10.668392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.669069 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.669980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.670325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:10.670706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.670706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:10.671258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:10.671746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:10.672335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:10.672335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:10.672335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:15.152797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:15.153557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:15.153683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:15.154706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:15.189957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:15.190398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.191661 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.193223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.194054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:15.194712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:15.195530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.196910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:15.196910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:15.196910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.196910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:15.196910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.196910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:15.625093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:15.625754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:15.625921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:15.627074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:15.627439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.628838 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.630023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:15.631612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:15.632373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.633385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:15.633385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:15.633385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.633385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:15.633385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.633385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:15.642146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:15.642945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:15.643061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:15.643213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:15.643820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:15.644100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.645091 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:15.646564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.647106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.648100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:15.648534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:15.648919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.648919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:15.648919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.648919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:15.649968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.650405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.650969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:15.656491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:15.657558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:15.657558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:15.657558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:15.657964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:15.658365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.659241 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:15.660294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.661479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.661920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:15.662632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:15.663352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:15.663976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.664822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:15.666919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:15.667456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:15.667593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:15.667865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:15.668359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:15.668810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:15.669013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.669868 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.670982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.671358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:15.671734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.671734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:15.672149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:15.673476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:15.674645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:15.674645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:15.674645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:20.153617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:20.154357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:20.154357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:20.155099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:20.156421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=580,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:31:20.192270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:20.192662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.193697 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.194806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.195451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:20.196280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:20.197080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.198104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:20.198104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:20.198104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.198104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:20.198104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.198104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:20.625938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:20.627145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:20.627285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:20.628424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:20.628850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.630270 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.633525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.634568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:20.634975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.634975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:20.635041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:20.635519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.636069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:20.636069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.636069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:20.642734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:20.642977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:20.643057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:20.643168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:20.643777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:20.644164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.644920 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:20.645512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:20.646640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.647620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.648349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:20.648969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.649799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.650700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:20.657153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:20.657592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:20.657848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:20.658053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:20.658701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:20.659117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.659738 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:20.660979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.661275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.662482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:20.663163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:20.663950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:20.664703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.665684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:20.667420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:20.667696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:20.667850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:20.667943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:20.668467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:20.668919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:20.669116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.669955 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.670856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.671331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:20.671803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.671803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:20.672083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:20.672555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:20.673162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:20.673162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:20.673162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:25.154313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:25.155094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:25.155170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:25.156532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:25.195960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:25.196405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.197343 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.198426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.199039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:25.200392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:25.201170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.203034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:25.203034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:25.203034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.203034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:25.203034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.203034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:25.627363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:25.628193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:25.628409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:25.629603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:25.630010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.630991 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.632594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.633128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:25.633654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.633654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:25.634345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:25.635051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.635910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:25.635910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.635910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:25.643339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:25.643696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:25.643816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:25.643936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:25.644475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:25.644856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.645629 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:25.646418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.647004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.647489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:25.648080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:25.648858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.649587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.650788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:25.657557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:25.657850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:25.657961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:25.658091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:25.658642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:25.659099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.659957 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:25.661120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.661603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.662522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:25.662864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:25.663235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.663235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:25.663235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.663235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.664544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:25.665425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:25.666381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.666381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:25.666381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.666381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:25.668190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:25.668535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:25.668649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:25.668833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:25.669442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:25.669971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:25.670260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.671159 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.672034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.672635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:25.673026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.673026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:25.673094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:25.673541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:25.674069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:25.674069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:25.674069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:30.155051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:30.155888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:30.155888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:30.157347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:30.186848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:30.187487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.188503 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.189263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.189651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:30.190035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:30.190515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.191135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:30.191135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:30.191135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.191135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:30.191135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.191135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:30.628522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:30.629026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:30.629200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:30.630163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:30.630401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.631193 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.632787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.633678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:30.634536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:30.635595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.636606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:30.636606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:30.636606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.636606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:30.636606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.636606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:30.643974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:30.644306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:30.644503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:30.644913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:30.645520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:30.645958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.646946 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:30.648650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.649143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.650114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:30.650513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:30.650933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.650933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:30.650933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.650933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:30.652114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.652586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.653138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:30.657909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:30.658154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:30.658282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:30.658410 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:30.659205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:30.659608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.660381 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:30.661450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.661963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.663180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:30.663745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:30.664159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.664159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:30.664159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.664159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.665156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:30.665637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:30.666280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.666280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:30.666280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.666280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:30.668679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:30.669078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:30.669254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:30.669495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:30.670088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:30.670359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:30.670497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.671189 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.672114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.672553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:30.673029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.673029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:30.673255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:30.673707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:30.674320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:30.674320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:30.674320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:35.158802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:35.158802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:35.158802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:35.158802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.188845 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.190676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:35.191421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:35.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.196020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:35.196020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:35.196020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.196020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:35.196020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.196020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:35.629199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:35.629966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:35.630250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.635846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:35.636813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:35.637458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.638041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.640087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:35.645032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:35.645032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:35.645032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:35.645146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:35.645595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:35.645838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.646553 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:35.648277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.649171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.649838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:35.650311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:35.650847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.650847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:35.650847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.650847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:35.651664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.652335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.653030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:35.658334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:35.658844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:35.658844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:35.659000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:35.659435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:35.659976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.660799 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:35.661572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.662390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.663144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:35.663772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:35.664679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:35.665824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:35.670097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:35.671180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:35.671180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:35.671180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:35.671180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:35.671180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.671575 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.673229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.673957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:35.674631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.674631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:35.674966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:35.675587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:35.676356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:35.676356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:35.676356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:40.156678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:40.157426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:40.157426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:40.158013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:40.191228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:40.191766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.192668 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.193958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:40.195072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.195072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:40.195452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:40.196092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.196520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:40.196520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.196520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:40.629937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:40.630430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:40.630523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:40.631255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:40.631481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.632108 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.632879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.633582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:40.634239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:40.634708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.635418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:40.635418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:40.635418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.635418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:40.635418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.635418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:40.644885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:40.645559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:40.645559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:40.645559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:40.645987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:40.646422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.646855 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:40.647738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:40.648841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.649670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.650711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:40.651352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.653249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.654210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:40.658774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:40.658982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:40.659069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:40.659169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:40.659981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:40.660339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.661009 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:40.661542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.661876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.662203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:40.662703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:40.663054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:40.663880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.665139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:40.669583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:40.669818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:40.669898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:40.669997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:40.670604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:40.671156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:40.671376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.672169 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.672749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.673155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:40.673546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:40.674007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:40.674583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:40.674583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:40.674583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:40.674583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:40.674583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:40.674583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:45.157646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:45.158718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:45.158718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:45.159633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:45.189550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:45.190009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:45.192511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:45.192912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.193202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:45.193497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.193933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:45.665437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:45.665437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:45.665437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:45.666288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:45.666560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:45.667643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:45.668031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.668880 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.671006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.671308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:45.671733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:45.672129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:45.672544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:45.676105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:45.676254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.678872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.679951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.682121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:45.682853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:45.683652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.683652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:45.683652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.683652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:45.685602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.686076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:45.686480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:45.687333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:45.687743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.688493 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:45.689569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.690246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:45.691458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:45.691860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.691860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:45.691860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.691860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.693181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:45.693571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:45.693965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:45.694249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:45.694430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:45.696417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:45.696773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.697071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:45.697354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:45.697760 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:50.160315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:50.161711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:50.162001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:50.163526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:50.193961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:50.194395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.195277 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.196010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.196613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:50.197397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:50.198697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.200113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:50.200113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:50.200113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.200113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:50.200113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.200113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:50.666915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:50.668056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:50.668056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:50.668574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:50.668930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.669739 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.670784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.671480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:50.672195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:50.673041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:50.674083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.674460 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:50.675152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.675602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.675984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:50.676490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:50.677226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.677948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.678576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:50.685621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:50.685917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:50.686023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:50.686141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:50.686704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:50.687235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.688274 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:50.689440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.690088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.690950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:50.692086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:50.692725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.692725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:50.692725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.692725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.693138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:50.695368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:50.696513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:50.697401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.698615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.699618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:50.700173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:50.700624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:50.701506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:50.701506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:50.701506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:55.161536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:55.162129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:55.162336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:55.163650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:55.202651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:55.202929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.203611 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.205158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:55.205467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.205467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:55.205833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:55.206222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.206635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:55.206635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.206635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:55.667611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:55.668190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:55.668409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:55.669523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:55.669947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.670903 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.672664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.673465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:55.674400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.674400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:55.675292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:55.675862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:55.676497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:31:55.677387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:55.677387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.677939 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:55.678789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.679874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.680574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:55.681488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:55.682243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.682867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.683297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:55.685975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:55.686127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:55.686223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:55.686290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:55.686712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:55.687094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.687720 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:31:55.688272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.688703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.689022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:55.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:55.689791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:55.690196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.690702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:31:55.693657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:31:55.693953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:31:55.694144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:31:55.694379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:31:55.694995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:55.695180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:31:55.695383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.696129 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.697268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.698586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:31:55.699093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:31:55.699472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:31:55.699974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:31:55.699974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:31:55.699974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:31:55.699974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:31:55.699974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:31:55.699974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:00.162376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:00.163284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:00.163284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:00.164495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:00.193638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:00.193951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.194579 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.196274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:00.196974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:00.197920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.198964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:00.198964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:00.198964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.198964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:00.198964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.198964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:00.668295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:00.668864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:00.669065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:00.669989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:00.670324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.671207 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.673104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.673863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:00.674463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:00.675194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:00.676160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:00.676524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:00.676602 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:00.677306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:00.677995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:00.678743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.678961 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:00.679679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.680339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.680642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:00.681115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:00.681406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.681406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:00.681406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.681406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:00.681882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.682328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.683185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:00.686255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:00.686522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:00.686637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:00.686779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:00.687137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:00.687341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.687775 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:00.688467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.688951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.689392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:00.689807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:00.690314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.690314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:00.690314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.690314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.690646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:00.691181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:00.692066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.692066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:00.692066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.692066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:00.693973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:00.694182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:00.694299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:00.694456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:00.694729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:00.694980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:00.695091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.695554 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.696317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.696679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:00.697088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.697088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:00.697271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:00.697726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:00.698360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:00.698360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:00.698360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:05.163077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:05.163944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:05.164068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:05.165198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:05.199463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:05.199903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.200799 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.201629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.202308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:05.203011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:05.203795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.204651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:05.204651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:05.204651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.204651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:05.204651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.204651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:05.669001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:05.669472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:05.669681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:05.670579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:05.670965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.671929 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.673387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.674191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:05.674831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.674831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:05.675138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:05.675859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.676689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:05.676689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.676689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:05.676689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:05.676901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:05.676901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:05.677014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:05.677486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:05.677863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.678721 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:05.679635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.680399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.680877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:05.681592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:05.682226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.683095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.683626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:05.686584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:05.686837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:05.686961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:05.687086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:05.687396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:05.687659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.688169 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:05.688927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.689358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.689814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:05.690156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:05.690598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.690598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:05.690598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.690598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.690998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:05.691477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:05.692028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.692028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:05.692028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.692028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:05.694244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:05.694656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:05.694909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:05.695201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:05.695659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:05.696082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:05.696280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.697236 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.699173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.699691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:05.700161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.700161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:05.701206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:05.701951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:05.702928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:05.702928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:05.702928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:10.164063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:10.164859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:10.165064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:10.167886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:10.201849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:10.202113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.202859 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.203979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.204412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:10.204795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.204795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:10.205468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:10.206310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.207052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:10.207052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.207052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:10.670021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:10.670393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:10.670560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:10.671817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:10.672233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.673157 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.674579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.675243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:10.675861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.675861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:10.676386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:10.677026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.677602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:10.677602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.677602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:10.678145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:10.678387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:10.678551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:10.678738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:10.679401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:10.679704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.680518 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:10.681460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:10.682555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.683530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.684165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:10.685232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.685833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:10.687543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:10.688609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:10.688609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:10.688894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:10.689325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.690311 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:10.691917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.692667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.693216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:10.694026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:10.694990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:10.696020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:10.697196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:10.698017 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.698582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:10.699202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:10.699735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:10.700195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:10.700651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:10.700651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:10.700651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:10.700651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:10.700651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:10.700651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:15.164858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:15.165587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:15.165587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:15.166861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:15.208352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:15.208775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:15.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:15.212288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.212542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:15.212833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.213284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:15.671385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:15.672851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:15.672851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:15.673355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:15.673651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:15.677834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:15.678625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.679294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:15.679294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:15.679294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.679294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:15.679294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.679294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:15.679978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:15.680883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.681889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:15.681889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.681889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:15.682479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:15.683221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:15.683221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:15.683221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:15.683692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:15.684031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.686688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:15.687110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.687441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.687705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:15.688030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:15.688339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:15.688787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:15.689209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:15.689362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:15.689533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:15.689968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:15.690227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.692833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:15.693240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.693530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.693792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:15.694050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:15.694360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.694724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:15.695117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:15.695401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:15.695548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:15.695739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:15.696139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:15.696441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:15.696569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:15.698827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:15.699421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.699959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:15.700465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:15.701083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:20.165601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:20.166267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:20.166267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:20.167395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:20.184477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=248 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:32:20.200337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:20.200973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.202144 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.203251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.203638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:20.204012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.204012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:20.204437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:20.204970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.205491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:20.205491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.205491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:20.672217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:20.672665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:20.672944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:20.674930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:20.674930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.675528 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.677569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.678289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:20.678954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.678954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:20.679495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:20.680322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.681230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:20.681230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.681230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:20.682929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:20.683215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:20.683378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:20.683555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:20.684430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:20.684790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.685632 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:20.686476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.687117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.687436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:20.687878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:20.688376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.688376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:20.688376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.688376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:20.688585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.689131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:20.689686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:20.689960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:20.690003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:20.690148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:20.690445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:20.690722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.691222 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:20.692050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.692560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.693012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:20.693342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:20.693814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.693814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:20.693814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.693814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.694158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:20.694728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:20.695298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.695298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:20.695298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.695298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:20.695593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:20.695982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:20.696058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:20.696158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:20.697346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:20.698228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:20.698479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.699432 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.700464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.700945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:20.701689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:20.702268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:20.702892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:20.702892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:20.702892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:20.702892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:20.702892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:20.702892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:25.166248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:25.166706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:25.166807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:25.167669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:25.198111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:25.198431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.199331 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.200440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.201222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:25.202094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:25.202854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.203885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:25.203885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:25.203885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.203885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:25.203885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.203885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:25.673034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:25.673604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:25.673716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:25.674386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:25.674555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.675212 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.676141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.676694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:25.677286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:25.678030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.679330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:25.679330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:25.679330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.679330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:25.679330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.679330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:25.683353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:25.683607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:25.683723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:25.683882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:25.684203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:25.684416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.684918 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:25.685649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.686154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.686483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:25.686980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:25.687353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.687353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:25.687353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.687353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:25.687552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.687996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.688479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:25.689910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:25.690135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:25.690248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:25.690369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:25.690653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:25.690915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.691364 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:25.692281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.692803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.693160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:25.693461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:25.693927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.693927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:25.693927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.693927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.694276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:25.694963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:25.695606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.695606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:25.695606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.695606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:25.695936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:25.696192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:25.696299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:25.696449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:25.696715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:25.696945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:25.697054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.697518 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.698329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.698856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:25.699311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.699311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:25.699394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:25.699815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:25.700305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:25.700305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:25.700305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:30.167099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:30.168224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:30.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:30.169027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:30.198633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:30.198973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.199652 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.200457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.201015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:30.201585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:30.202241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.203128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:30.203128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:30.203128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.203128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:30.203128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.203128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:30.674109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:30.674602 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:30.674777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:30.675897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:30.676097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.676739 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.677991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.678621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:30.679373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:30.680345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.681325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:30.681325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:30.681325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.681325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:30.681325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.681325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:30.683784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:30.684346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:30.684623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:30.684975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:30.685510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:30.685857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.686787 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:30.688736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.690080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.690863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:30.691733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:30.692451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.693184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:30.693964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:30.694488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:30.694488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:30.694488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:30.694488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.695175 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:30.696109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.696720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.697450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:30.697992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:30.698508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.698508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:30.698508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.698508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:30.698508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.699209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:30.699973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:30.700606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:30.700889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:30.700954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.701745 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.702716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.703093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:30.703431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.703431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:30.703855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:30.704261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:30.704786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:30.704786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:30.704786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:35.168285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:35.168995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:35.168995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:35.169384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:35.194344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:35.194600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.195267 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.196030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.196554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:35.197161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:35.197824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.198668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:35.198668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:35.198668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.198668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:35.198668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.198668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:35.677070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:35.677886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:35.678113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:35.679379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:35.679789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.680979 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.682493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.683063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:35.683689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.683689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:35.685372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:35.685771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.686148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:35.686148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.686148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:35.686148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:35.686148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:35.686391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:35.686587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:35.687118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:35.687491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.688237 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:35.689520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.690050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.691007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:35.691409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:35.691822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.692304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.692933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:35.693312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:35.693312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:35.693409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:35.693551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:35.693930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:35.694222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.694723 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:35.695243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.695690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.696018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:35.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:35.696822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:35.697293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.697926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:35.699489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:35.699690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:35.699771 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:35.699909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:35.700201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:35.700486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:35.700647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.701255 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.701832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.702229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:35.702680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:35.703124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:35.703646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:35.703646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:35.703646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:35.703646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:35.703646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:35.703646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:40.168883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:40.169531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:40.169839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:40.171153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:40.201787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:40.202335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.203389 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.204427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.205075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:40.205642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:40.206368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.207226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:40.207226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:40.207226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.207226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:40.207226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.207226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:40.677707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:40.678191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:40.678321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:40.679275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:40.679634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.681887 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.682932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.683245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:40.683552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.683552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:40.684105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:40.684506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.684970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:40.684970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.684970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:40.686044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:40.686260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:40.686376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:40.686504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:40.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:40.687380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.687831 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:40.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.689044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.689529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:40.689847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:40.690149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.690149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:40.690149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.690149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:40.690592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.690987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.691392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:40.693349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:40.693561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:40.693676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:40.693827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:40.694186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:40.694528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.695104 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:40.695887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.696234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.696830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:40.697117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:40.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:40.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.697941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:40.698423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:40.698869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.698869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:40.698869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.698869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:40.699739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:40.699965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:40.700078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:40.700227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:40.700704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:40.701090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:40.701266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.701999 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.702884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.703178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:40.703470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.703470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:40.703888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:40.704270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:40.704728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:40.704728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:40.704728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:45.169545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:45.170346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:45.170346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:45.171646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:45.204445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:45.205025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.206053 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.206772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.207170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:45.207550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:45.208031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.208727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:45.208727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:45.208727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.208727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:45.208727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.208727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:45.678237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:45.678650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:45.678650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:45.679276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:45.679543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.680129 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.681015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.681579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:45.682144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:45.682859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.683641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:45.683641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:45.683641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.683641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:45.683641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.683641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:45.686421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:45.686706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:45.686970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:45.686970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:45.687510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:45.687876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.688625 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:45.689476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:45.690433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.691467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.692310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:45.693191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.694199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:45.695387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:45.695780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:45.695780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:45.695780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.695913 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:45.696558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.697162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.697433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:45.697795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:45.698173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.698173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:45.698173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.698173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.698555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:45.699044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:45.699527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.699527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:45.699527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.699527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:45.700199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:45.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:45.700927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:45.701223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:45.701782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:45.702224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:45.702429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.703332 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.704829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.705222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:45.705698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.705698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:45.706719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:45.707351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:45.708222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:45.708222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:45.708222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:50.170232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:50.171133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:50.171133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:50.172546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:50.199448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:50.199924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.200912 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.201714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.202446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:50.203137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:50.203988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.204908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:50.204908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:50.204908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.204908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:50.204908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.204908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:50.678870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:50.679518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:50.679713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:50.680684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:50.680940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.681600 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.683217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.683943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:50.684635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.684635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:50.685005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:50.686024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.686860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:50.686860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.686860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:50.687351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:50.687947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:50.688164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:50.688372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:50.688832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:50.689037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.689578 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:50.690460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.690905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.691314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:50.691856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:50.692225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.692225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:50.692225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.692225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:50.692527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.693081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.693655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:50.694022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:50.694375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:50.694481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:50.694610 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:50.694979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:50.695196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.695688 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:50.696506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.696995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.697412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:50.697884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:50.698353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.698353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:50.698353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.698353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:50.699206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:50.699672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.699672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:50.699672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.699672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:50.700690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:50.700980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:50.701115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:50.701318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:50.701659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:50.701917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:50.702031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.702517 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.703401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:50.704239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.704239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:50.704452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:50.704903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:50.705452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:50.705452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:50.705452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:55.171192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:55.172336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:55.172336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:55.173621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:55.210119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:55.210563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.211728 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.213048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:55.213454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:55.213962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.214511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:55.214511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:55.214511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.214511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:55.214511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.214511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:55.680008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:55.680527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:55.680860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:55.681975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:55.682313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.683240 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.684739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.685442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:55.686283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.686283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:55.686485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:55.686982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.687512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:55.687512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.687512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:55.688021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:55.688306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:55.688431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:55.688561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:32:55.689079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:55.689387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.690151 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:55.690934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.691422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.691783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:55.692174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:55.692569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.692569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:55.692569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.692569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:55.692957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.693410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.693939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:55.694490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:55.694898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:55.695067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:55.695241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:55.695709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:55.696116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.696819 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:32:55.697643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.698074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.698614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:55.698993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:55.699372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.699372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:55.699372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.699372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.699733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:55.700424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:55.701005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.701005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:55.701005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.701005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:32:55.701307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:32:55.701563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:32:55.701728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:32:55.701939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:32:55.702408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:55.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:32:55.702988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.703670 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.704653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.705076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:32:55.705479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.705479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:32:55.706109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:32:55.706597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:32:55.707206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:32:55.707206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:32:55.707206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:00.172336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:00.173168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:00.173313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:00.174709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:00.205627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:00.206138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.207247 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.208137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.208706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:00.209320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:00.210161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.211071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:00.211071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:00.211071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.211071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:00.211071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.211071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:00.680505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:00.681418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:00.681627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:00.682774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:00.683930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.684219 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.686055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.686883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:00.687544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.687544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:00.687945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:00.688683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:00.689541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:00.689915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:00.690095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.691023 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:00.691849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.692543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.693008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:00.693541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:00.694189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.694896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:00.695723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:00.696504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:00.696504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:00.696504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:00.696504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.697325 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:00.698158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.698493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.699150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:00.699386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:00.699697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.699697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:00.699697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.699697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.700300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:00.700653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:00.701054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.701054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:00.701054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.701054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:00.701790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:00.702159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:00.702287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:00.702487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:00.702869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:00.703104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:00.703254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.703840 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.704753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.705045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:00.705326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.705326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:00.705837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:00.706242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:00.706713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:00.706713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:00.706713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:05.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:05.173648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:05.173648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:05.174451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:05.209697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:05.210254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.211358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.212487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.212998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:05.213368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.213368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:05.213635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:05.214108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.214634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:05.214634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.214634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:05.681270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:05.681972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:05.682194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:05.683477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:05.684031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.685217 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.686986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.687793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:05.688316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.688316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:05.688591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:05.689980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:05.690740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.691173 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:05.691897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.692822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.693319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:05.694020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:05.694417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.694859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.695395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:05.695679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:05.695876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:05.696032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:05.696203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:05.696575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:05.696958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.697766 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:05.698683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.700370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:05.700880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:05.701298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.701298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:05.701298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.701298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.701402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:05.701923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:05.702392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.702392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:05.702392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.702392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:05.702392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:05.702539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:05.703128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:05.703128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:05.703806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:05.704071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:05.704308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.705150 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.706089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.706663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:05.706978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.706978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:05.707262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:05.707647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:05.708017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:05.708017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:05.708017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:10.173901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:10.174289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:10.174404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:10.175658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:10.208457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:10.208910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.209714 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.210586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.211302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:10.211951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:10.212727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.213689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:10.213689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:10.213689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.213689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:10.213689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.213689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:10.681957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:10.682413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:10.682664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:10.683472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:10.683817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.684794 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.686237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.686946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:10.687514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.687514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:10.687617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:10.688482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.689506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:10.689506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.689506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:10.689506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:10.689721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:10.689799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:10.689930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:10.690406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:10.690854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.691541 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:10.692270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.692701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.693096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:10.693442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:10.693822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.693822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:10.693822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.693822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:10.694091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.694545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.695056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:10.695975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:10.696250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:10.696361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:10.696482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:10.696937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:10.697309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.697975 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:10.698678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.699114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.699537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:10.699898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:10.700271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.700271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:10.700271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.700271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.700568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:10.701059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:10.701557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.701557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:10.701557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.701557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:10.702893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:10.703037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:10.703144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:10.703460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:10.703692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:10.704128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:10.704327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.705173 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.706323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:10.706682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.706682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:10.706995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:10.707467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:10.708010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:10.708010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:10.708010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:15.174171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:15.178656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:15.178656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:15.178656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:15.210990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:15.211242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.211903 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.214851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:15.215191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:15.215645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.216188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:15.216188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:15.216188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.216188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:15.216188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.216188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:15.682630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:15.683438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:15.683627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:15.684563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:15.684972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.685863 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.687450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.688189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:15.688722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.688722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:15.689322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:15.690304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:15.692353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.692976 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:15.693863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.694386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:15.695614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:15.696117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.696117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:15.696117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.696117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:15.696601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.697074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:15.697815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:15.698285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:15.698285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:15.698285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.698398 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:15.698983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.699486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.699814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:15.700254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:15.700786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:15.701308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.701854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:15.703196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:15.703469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:15.703559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:15.703672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:15.704196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:15.704560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:15.704728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.705498 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.706476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.706829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:15.707065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.707065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:15.707517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:15.707910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:15.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:15.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:15.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:20.175912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:20.176436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:20.176564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:20.177431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:20.192571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=236 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:33:20.211293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:20.211539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.212121 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.212938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.213266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:20.213638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.213638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:20.213713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:20.214163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.214667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:20.214667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.214667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:20.683575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:20.684098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:20.684235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:20.685572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:20.686131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.687537 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.688751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.689426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:20.690018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:20.690673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:20.691485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:20.692163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:20.692466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.693512 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:20.694367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.695149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.695572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:20.696205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:20.697555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.698696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:20.699458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.699870 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:20.700269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.700747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.701098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:20.701456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:20.701841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.701841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:20.701841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.701841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.702336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:20.703008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:20.703527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.703527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:20.703527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.703527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:20.703807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:20.704082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:20.704215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:20.704582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:20.705144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:20.705624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:20.705870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.706683 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.707560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.707938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:20.708378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.708378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:20.708650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:20.709109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:20.709675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:20.709675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:20.709675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:25.176676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:25.177516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:25.177516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:25.178696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:25.214930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:25.215387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.216434 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.217725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.218277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:25.218846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:25.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.220391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:25.220391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:25.220391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.220391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:25.220391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.220391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:25.684250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:25.684985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:25.685186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:25.686284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:25.686658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.687659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.689244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.689517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:25.689753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.689753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:25.691130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:25.692608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:25.694487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.695156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.695638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.696370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:25.696677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:25.697602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:25.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:25.700520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.700949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.701353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:25.701660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:25.702037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.702037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:25.702037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.702037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.702360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:25.702984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:25.703551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.703551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:25.703551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.703551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:25.704189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:25.704374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:25.704481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:25.704622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:25.705069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:25.705424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:25.705600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.706330 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.707161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.707455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:25.707698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.707698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:25.708106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:25.708489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:25.710101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:25.710101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:25.710101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:30.177338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:30.178061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:30.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:30.179675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:30.210563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:30.210996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.211801 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.212678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.213286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:30.214123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:30.214804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.215596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:30.215596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:30.215596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.215596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:30.215596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.215596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:30.685010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:30.685456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:30.685723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:30.686814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:30.687159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.688165 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.689922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.690270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:30.690596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.690596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:30.691599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:30.692029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.692545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:30.692545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.692545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:30.692545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:30.692746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:30.692922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:30.693113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:30.693653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:30.694060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.694846 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:30.695632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.696077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.696468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:30.697014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:30.697347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.697347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:30.697347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.697347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:30.697556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.698225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:30.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:30.699442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:30.699442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:30.699442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.699826 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:30.700571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.700967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.701435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:30.701732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:30.702086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.702086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:30.702086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.702086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.702613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:30.703367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:30.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:30.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.703812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:30.704631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:30.704964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:30.705073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:30.705230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:30.705571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:30.705986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:30.706188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.706975 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.707848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.708082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:30.708303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.708303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:30.708949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:30.709313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:30.709642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:30.709642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:30.709642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:35.178052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:35.178671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:35.178929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:35.181497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:35.211223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:35.211581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.212519 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.214250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.215253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:35.216078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:35.217072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.218273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:35.218273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:35.218273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.218273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:35.218273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.218273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:35.685822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:35.686407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:35.686625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:35.687692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:35.688029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.688845 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.689684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.690066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:35.690436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.690436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:35.690772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:35.691125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.691463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:35.691463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.691463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:35.692655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:35.692869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:35.692997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:35.693114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:35.693450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:35.693670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.694187 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:35.694835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.695227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.695900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:35.696253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:35.696619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.696619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:35.696619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.696619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:35.696993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.697313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.697654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:35.698576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:35.698900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:35.699091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:35.699205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:35.699732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:35.700166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.701032 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:35.701823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.702433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.703120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:35.703539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:35.704055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.704055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:35.704055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.704055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.704307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:35.704913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:35.705564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:35.706659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:35.706913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:35.707034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.707629 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.709287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.709775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:35.710305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.710305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:35.711472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:35.712074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:35.712726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:35.712726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:35.712726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:40.178684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:40.179431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:40.179710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:40.181099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:40.218297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:40.218745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.219720 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.221034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.221622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:40.222229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.222229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:40.222299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:40.222751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.223247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:40.223247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.223247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:40.686893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:40.687525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:40.687648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:40.688441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:40.688777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.689610 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.690746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.691176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:40.691643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.691643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:40.692125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:40.692656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.693203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:40.693203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.693203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:40.693203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:40.693203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:40.693372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:40.693428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:40.693940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:40.694213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.694925 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:40.695785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.696469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.696965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:40.697495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:40.698169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.699170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:40.700144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.700928 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:40.701387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.701956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.702228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:40.702609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:40.703032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.703032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:40.703032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.703032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.703230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:40.703784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:40.704692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.704692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:40.704692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.704692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:40.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:40.705464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:40.705578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:40.705797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:40.706081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:40.706310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:40.706435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.706956 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.707810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.708498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:40.708908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:40.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:40.709917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:40.709917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:40.709917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:40.709917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:40.709917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:40.709917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:45.179510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:45.180628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:45.180965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:45.182376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:45.215319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:45.215601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.216277 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.217031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.217594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:45.218224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:45.219020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.219905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:45.219905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:45.219905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.219905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:45.219905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.219905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:45.687882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:45.688575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:45.688867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:45.689742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:45.690337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.691024 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.692556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.693018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:45.693469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.693469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:45.694391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:45.694901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.695467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:45.695467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.695467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:45.695467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:45.695467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:45.695665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:45.695705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:45.696178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:45.696390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.696939 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:45.698056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.698568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.699176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:45.699588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:45.699996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.699996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:45.699996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.699996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:45.700793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.701288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:45.701961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:45.702546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:45.702718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:45.703076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.703705 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:45.704787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.705231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:45.706567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:45.707046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.707046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:45.707046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.707046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:45.707046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.708217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:45.708728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:45.709805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:45.710184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:45.710184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.710713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.711869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.712259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:45.712622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.712622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:45.713237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:45.713808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:45.714383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:45.714383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:45.714383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:50.180383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:50.181037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:50.181234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:50.182488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:50.212463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:50.212842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.213130 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.214018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.214432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:50.214803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.214803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:50.214972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:50.215415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.215891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:50.215891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.215891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:50.688539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:50.689031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:50.689152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:50.689685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:50.689873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.690400 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.691332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.692142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:50.693020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:50.693929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.695282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:50.695282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:50.695282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.695282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:50.695282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.695282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:50.695949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:50.696254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:50.696372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:50.696487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:50.696851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:50.697054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.697534 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:50.698267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.698673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.699116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:50.699426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:50.699782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.699782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:50.699782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.699782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:50.700140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.700530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.700955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:50.701619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:50.701896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:50.702006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:50.702132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:50.702399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:50.702628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.703068 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:50.703775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.704192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.704642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:50.704958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:50.705324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.705324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:50.705324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.705324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.705750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:50.706202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:50.706693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.706693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:50.706693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.706693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:50.708695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:50.708996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:50.709103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:50.709265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:50.709539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:50.709794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:50.709913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.710363 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.711213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.711543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:50.711889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.711889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:50.712163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:50.712612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:50.713088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:50.713088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:50.713088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:55.181067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:55.181727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:55.181845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:55.182553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:55.222036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:55.222249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.223059 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.224075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.224695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:55.225350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:55.226088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.227013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:55.227013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:55.227013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.227013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:55.227013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.227013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:55.689237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:55.689785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:55.690086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:55.690847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:55.691166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.691969 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.693423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.693809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:55.694137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.694137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:55.695038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:55.695454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.695944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:55.695944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.695944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:55.696420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:55.696836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:55.696949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:55.697069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:33:55.697429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:55.697686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.698377 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:55.699437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.699882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.700694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:55.701043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:55.701413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.701413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:55.701413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.701413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:55.702254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.702671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:55.703139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:55.703405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:55.703640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:55.704226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.705290 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:33:55.705981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.706374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.706887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:55.707207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:55.707561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.707561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:55.707561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.707561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.707934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:55.708810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:33:55.709706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:33:55.710128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:55.710191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:33:55.710253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.710678 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.711455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.711774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:33:55.712066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.712066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:33:55.712437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:33:55.713643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:33:55.714462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:33:55.714462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:33:55.714462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:00.181706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:00.182584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:00.182647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:00.183989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:00.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:00.211934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.212180 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.212849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.213394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:00.213983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:00.214810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.216654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:00.216654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:00.216654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.216654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:00.216654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.216654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:00.689844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:00.690578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:00.690775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:00.691611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:00.691844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.692651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.694501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.695489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:00.696009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.696009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:00.696505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:00.697813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:00.698538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.698817 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:00.699446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:00.700551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.701528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.702065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:00.702440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.702883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:00.704235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:00.704919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:00.704919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:00.704919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.705201 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:00.705869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.706298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.707125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:00.707751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:00.708478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:00.709428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:00.710164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:00.710499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:00.710499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:00.710550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:00.710903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:00.711031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.711501 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.712388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.712629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:00.712887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.712887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:00.713487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:00.713822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:00.714184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:00.714184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:00.714184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:05.182451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:05.183122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:05.183188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:05.184442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:05.219972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:05.220196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.220848 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.221997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.222551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:05.223163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:05.224020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.225271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:05.225271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:05.225271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.225271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:05.225271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.225271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:05.690500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:05.691103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:05.691324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:05.692139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:05.692389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.693310 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.694721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.695314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:05.695893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.695893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:05.696352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:05.697222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.698230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:05.698230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.698230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:05.698230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:05.698431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:05.698705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:05.698944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:05.699372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:05.699651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.700516 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:05.701963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.702889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.703447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:05.704289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:05.704969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.705383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:05.705966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.706392 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:05.706806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:05.707298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:05.707911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:05.708331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:05.708749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.709454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:05.710126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:05.710195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:05.710298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:05.710414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:05.710753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:05.711013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:05.711136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.711582 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.712202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.712649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:05.713165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:05.713655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:05.714249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:05.714249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:05.714249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:05.714249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:05.714249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:05.714249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:10.183109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:10.183820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:10.183820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:10.185335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:10.214705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:10.215020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.215908 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.217582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.217954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:10.218336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.218336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:10.219571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:10.220023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.220509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:10.220509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.220509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:10.691172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:10.691677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:10.692036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:10.693010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:10.693334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.694022 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.695014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.695420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:10.695804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.695804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:10.696087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:10.696555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.697097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:10.697097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.697097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:10.698469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:10.698744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:10.698877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:10.699001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:10.699673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:10.700031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.700782 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:10.701707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.702158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.702560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:10.703164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:10.703575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.704415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.705320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:10.707893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:10.708197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:10.708314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:10.708447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:10.709132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:10.709717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.710719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.710992 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:10.711693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.712111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.712557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:10.712922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:10.713322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.713322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:10.713322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.713322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.713649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:10.714114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:10.714592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:10.714840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:10.714840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.715217 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.715989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.716359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:10.717011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.717011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:10.717082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:10.717526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:10.718025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:10.718025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:10.718025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:15.183804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:15.184366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:15.184497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:15.185747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:15.222654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:15.222998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.223821 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.225599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:15.226291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.226291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:15.226411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:15.227367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.228857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:15.228857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.228857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:15.691833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.696426 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.696952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.697418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:15.697927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.697927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:15.698550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:15.699178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:15.699914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:15.700204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:15.700537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.701406 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:15.702595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.703267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.703966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:15.704451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:15.704949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.704949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:15.704949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.704949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:15.705619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.706191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.706930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:15.708614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:15.708972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:15.709041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:15.709136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:15.709848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:15.710157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.710753 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:15.711373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.711924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.712272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:15.712684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:15.713206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:15.713695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:15.714361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:15.714805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:15.714805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:15.714805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:15.715144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:15.715417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:15.715538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.716153 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.717047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.717517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:15.717984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.717984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:15.718069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:15.718616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:15.719287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:15.719287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:15.719287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:19.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=dbad64b7-b3a7-40ad-b5d3-5c300a6b233f, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}, gateway_chassis=[], ha_chassis_group=71d9c044-6750-418f-be8e-8cb92dbbbbd2, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:53:d5:75', name='lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b', networks=['172.24.5.167/24', '2001:db8::248/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105368) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 05 01:34:20.184985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:20.186339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:20.186339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:20.186779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:20.200721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=239 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:34:20.225463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.225996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.226970 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.228064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.229062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:20.229869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:20.230822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.231962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:20.231962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:20.231962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.231962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:20.231962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.231962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:20.694196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:20.696801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:20.697008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:20.698887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.699144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.699316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.700241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.700610 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.701958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.702835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:20.703448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:20.704160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:20.704731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.705122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.705706 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:20.706638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.707322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.707989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:20.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:20.709157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.709157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:20.709157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.709157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:20.709416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.709952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:20.710632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:20.711006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:20.711089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.711352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.711890 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:20.712889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.713499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.714134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:20.714599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:20.715213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.715213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:20.715213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.715213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.715733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:20.716407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:20.717180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:20.717394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.717394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:20.717394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.717822 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.718749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.719288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:20.719845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.719845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:20.720185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:20.720805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:20.721228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:20.721228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:20.721228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:21.770478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=852db500-aaf9-4420-9e3e-a8153772992b, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-28e436bb-9153-4de9-91bf-b593e5bb4624', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a', 'neutron:subnet_ids': 'f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:23:e0:6e', name='lrp-2654f5b0-6ac8-4dbd-b55b-80899173f81c', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=105368) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 05 01:34:21.806771 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-28e436bb-9153-4de9-91bf-b593e5bb4624 synchronized; highest priority chassis db4f2cd9-f03d-427b-86b1-bf324a42ead8
May 05 01:34:21.807471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'neutron:router_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a'}}) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:21.807992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, chassis=db4f2cd9-f03d-427b-86b1-bf324a42ead8, priority=1, columns={}) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:21.808294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:21.808915 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event Traceback (most recent call last):
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2415, in link_network_ha_chassis_group
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     next(self.gen)
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     with self.create_transaction(
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     self.result = self.commit()
May 05 01:34:21.809956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise result.ex
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:21.810906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.event 
May 05 01:34:25.185881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:25.188790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:25.189023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:25.190311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:25.234081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.234574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.235616 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.237282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.237824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:25.238339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.238339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:25.238926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:25.240280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.241341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:25.241341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.241341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:25.698007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:25.698481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:25.698719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:25.699269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.699436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.699579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.700162 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.701222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.701683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:25.702231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.702231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:25.702808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:25.703717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:25.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.705570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.705969 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:25.706900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.708267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:25.709400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:25.710334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.711108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:25.711908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:25.712807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:25.713836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:25.714294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:25.714621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:25.714894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.715240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:25.716322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:25.716609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:25.716714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:25.716889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:25.717237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.717585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:25.717752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.718398 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.719088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.719400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:25.719730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.719730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:25.720175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:25.720803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:25.721536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:25.721536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:25.721536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:30.188950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:30.189953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:30.189953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:30.190338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:30.220926 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.221175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.224451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.224853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:30.225205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:30.225621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.226157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:30.226157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:30.226157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.226157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:30.226157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.226157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:30.698676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:30.699489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:30.699731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:30.700775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.701094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.701289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.702164 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.703753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.704494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:30.705053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.705053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:30.705739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:30.706358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.706859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:30.706859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.706859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:30.706859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:30.706859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:30.707013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:30.707091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:30.707550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.707939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.708628 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:30.709906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.710315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.711363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:30.711724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:30.712088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.712088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:30.712088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.712088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:30.712088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:30.712660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.713197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:30.713658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:30.713954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.713954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.714086 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:30.714771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.715161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.715742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:30.716058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:30.716398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.716398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:30.716398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.716398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.716867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:30.717327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:30.717796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:30.718038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.718152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:30.718354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.719174 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.720870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.721206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:30.721533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.721533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:30.722669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:30.723127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:30.723598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:30.723598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:30.723598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:35.189738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:35.190513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:35.190513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:35.191553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:35.233200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.233847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.234921 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.236063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.236670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:35.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:35.237946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.239532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:35.239532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:35.239532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.239532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:35.239532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.239532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:35.701019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:35.701019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:35.701019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:35.701019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.701019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.701019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.701733 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.702400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.702777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:35.703096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.703096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:35.703155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:35.703553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.704051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:35.704051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.704051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:35.706898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:35.707198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:35.707320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:35.707454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:35.708256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.708502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.709079 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:35.710175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.710654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.712668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:35.713119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:35.713527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.714104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:35.714635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:35.715082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:35.715082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:35.715154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.715465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.716030 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:35.719816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:35.720316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:35.720735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:35.721187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:35.721564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.722093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:35.722614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:35.722614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:35.722614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:35.722614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:35.722614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.722614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.722902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.723372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:35.723788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:35.724143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:35.724993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:40.190214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:40.191125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:40.191125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:40.192326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:40.227352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.227639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.228324 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.229185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.229648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:40.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:40.230643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.231302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:40.231302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:40.231302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.231302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:40.231302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.231302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:40.700327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:40.700801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:40.700980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:40.702175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.702521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.702769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.703777 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.705282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.705658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:40.706059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.706059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:40.707019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:40.707446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:40.708100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:40.708520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.708854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.709613 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:40.711044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.711483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.712052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:40.712395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:40.712796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.712796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:40.712796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.712796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:40.713103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.713531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:40.714038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:40.714329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:40.714329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.714654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.715429 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:40.716155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.716564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.717009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:40.717364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:40.717888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.717888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:40.717888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.717888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.718269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:40.718782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:40.719422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.719422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:40.719422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.719422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:40.719730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:40.719990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:40.720092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:40.720246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:40.720790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.721156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:40.721462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.722229 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.723061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.723502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:40.724040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:40.724559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:40.725224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:40.725224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:40.725224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:40.725224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:40.725224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:40.725224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:45.190725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:45.191316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:45.191527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:45.193329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:45.228911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.230338 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.231176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.231713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:45.232227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:45.232853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.233562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:45.233562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:45.233562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.233562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:45.233562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.233562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:45.701018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:45.701580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:45.702120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:45.703243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.703573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.703868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.705182 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.706908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.707502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:45.708057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.708057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:45.709252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:45.710142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:45.710959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:45.711326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.711326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.712219 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:45.713249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.713711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.714368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:45.714769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:45.715160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.715160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:45.715160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.715160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:45.715160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:45.715872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:45.716844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:45.717672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.717672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.717969 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:45.718771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.719210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.719945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:45.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:45.720649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.720649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:45.720649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.720649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.721247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:45.721751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:45.722268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:45.722516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.722516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:45.722592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.723060 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.723718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.724083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:45.724440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.724440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:45.724633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:45.725046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:45.725502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:45.725502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:45.725502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:50.191374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:50.192164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:50.192164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:50.192482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:50.236497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.236957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:50.240216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:50.240719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.241150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:50.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.242116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:50.702175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:50.702720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:50.703127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:50.704335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.704571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.704725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:50.708658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:50.709230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.709635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.710038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:50.710584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:50.711247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:50.711468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:50.711696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:50.712144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.712331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.715491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:50.715999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.716301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.716537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:50.716801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:50.717088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.717449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:50.717877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:50.718264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:50.718264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:50.718264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:50.720892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.721238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.722075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.722262 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:50.725306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:50.725852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:50.726396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:50.726857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:50.727300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.727982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:50.728647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:50.728647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:50.728647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:50.728647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.728647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:50.728647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.729160 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.730318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.730720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:50.731078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.731078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:50.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:50.732744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:50.733102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:50.733102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:50.733102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:55.191990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:55.195438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:55.195438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:55.195438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:55.230141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.230696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.231722 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.232192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.232464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:55.232732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.232732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:55.233248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:55.233602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.234005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:55.234005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.234005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:55.702965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:55.704388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:55.704388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:55.705135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.705420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.705580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.706225 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.707311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.707711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:55.708068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.708068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:55.708472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:55.708964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.709492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:55.709492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.709492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:55.711361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:55.711685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:55.711851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:55.711997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:34:55.712475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.712778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.713615 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:55.714745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.715176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.716004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:55.716340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:55.716669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.716669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:55.716669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.716669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:55.717601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.717966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.718326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:55.718476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:55.718883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:55.718970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:55.719166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:55.719665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.720052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.720823 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:34:55.722118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.722519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.723398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:55.723670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:55.723980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.723980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:55.723980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.723980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.724929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:55.725285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:55.725649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.725649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:55.725649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.725649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:34:55.725649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:34:55.725649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:34:55.725833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:34:55.726007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:34:55.727064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.727710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:34:55.728021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.728707 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.729337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.729751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:34:55.730156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:34:55.730652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:34:55.731191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:34:55.731191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:34:55.731191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:34:55.731191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:34:55.731191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:34:55.731191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:00.215732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:00.215732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:00.215732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:00.215732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:00.244217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.244663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.245648 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.246499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.247101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:00.247560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:00.248117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.248720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:00.248720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:00.248720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.248720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:00.248720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.248720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:00.703873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:00.705307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:00.705585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:00.707062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.707062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.707062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.707320 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.708416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.709049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:00.709395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.709395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:00.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:00.709883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.710219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:00.710219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.710219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:00.711905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:00.712222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:00.712344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:00.712471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:00.713073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.713073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.713315 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:00.714175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.714441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.715075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:00.715306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:00.715539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.715539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:00.715539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.715539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:00.716230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.716622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.717424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:00.718894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:00.719211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:00.719347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:00.719452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:00.719844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.720579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.721075 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:00.721848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.722555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.722795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:00.723074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:00.723402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.723402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:00.723402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.723402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.723967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:00.725030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:00.726241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:00.727512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.727512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:00.727512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.727737 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.728621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.729256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:00.729571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.729571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:00.729939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:00.730726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:00.731200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:00.731200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:00.731200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:05.194114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:05.194845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:05.195244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:05.196303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:05.232399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.232663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.233187 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.233903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.234470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:05.235049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:05.235804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.236524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:05.236524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:05.236524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.236524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:05.236524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.236524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:05.705105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:05.705575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:05.705781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:05.706694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.707053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.707393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.708226 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.709679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.710791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:05.711983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:05.713395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:05.715111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:05.715754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.716383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:05.717255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:05.717468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.717730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.718950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:05.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:05.720566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:05.720566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:05.720566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:05.720566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.720566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.721106 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:05.721851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.722452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.722820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:05.723287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:05.723867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.723867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:05.723867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.723867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.724058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:05.724841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:05.725177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.725177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:05.725177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.725177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:05.725980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:05.726153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:05.726267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:05.726418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:05.726640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.726859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:05.726983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.727384 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.728211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.728446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:05.728678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.728678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:05.729271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:05.729580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:05.729934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:05.729934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:05.729934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:10.195317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:10.196235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:10.196235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:10.197998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:10.246544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.247011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.248147 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.249503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.249963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:10.250818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:10.251273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.252092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:10.252092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:10.252092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.252092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:10.252092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.252092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:10.705655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:10.706017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:10.706197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:10.707193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.707548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.707822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.708580 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.709567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.710444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:10.710811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:10.711226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.711726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:10.711726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:10.711726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.711726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:10.711726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.711726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:10.713260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:10.713345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:10.713447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:10.713557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:10.714021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.714576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.715079 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:10.715796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.716307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.716580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:10.716981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:10.717357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.717357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:10.717357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.717357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:10.717739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.718934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:10.719518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:10.720097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:10.720097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:10.720097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:10.720287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.720621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.721101 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:10.722006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:10.723889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:10.726133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:10.726824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:10.727379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.728244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:10.729909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:10.729909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:10.729909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:10.729909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:10.729909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.729909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.730639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.731344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:10.731984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:10.732577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:10.733479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:15.208930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:15.208930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:15.208930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:15.208930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:15.236010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.236426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.237401 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.238449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.238708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:15.238973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.238973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:15.239506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:15.239863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.240206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:15.240206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.240206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:15.706224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:15.707172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:15.707172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:15.708436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.708846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.709071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.710217 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.712103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.712863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:15.713551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.713551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:15.714692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:15.715560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:15.716226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:15.716445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.716953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.717405 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:15.718491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:15.720030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.720674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.721179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:15.721657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.722202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:15.723654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.724581 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:15.725554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.726118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.726837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:15.727911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:15.728849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:15.729445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:15.730125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:15.730429 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:15.730671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.730671 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:15.730671 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.730671 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.731188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.731514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:15.731869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.731869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:15.732273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:15.732585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:15.732925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:15.732925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:15.732925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:20.196452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:20.196989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:20.197176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:20.198583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:20.208564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=256 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:35:20.230549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.230902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.231562 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:20.232946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.232946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:20.233120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:20.234409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.234890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:20.234890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.234890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:20.707378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:20.708090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:20.708222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:20.709196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.709530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.709810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.710649 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.711661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.712020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:20.712362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.712362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:20.712840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:20.713281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.713857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:20.713857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.713857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:20.715493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:20.715863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:20.715988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:20.716118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:20.716526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.716855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.717532 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:20.718313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.718701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.719193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:20.719535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:20.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:20.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.719950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:20.720276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.720770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.721253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:20.722312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:20.722617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:20.722731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:20.722915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:20.723299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.723739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.724429 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:20.725205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.725642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.726195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:20.726514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:20.726933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.726933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:20.726933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.726933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.727369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:20.727886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:20.728381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.728381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:20.728381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.728381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:20.728938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:20.729242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:20.729358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:20.729551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:20.729962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.730316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:20.730552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.731243 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.732246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.732689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:20.733087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.733087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:20.733379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:20.733887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:20.734404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:20.734404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:20.734404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:25.197092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:25.198434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:25.198434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:25.199274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:25.243104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.243528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.244432 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.245266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.245928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:25.246530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:25.247321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.248267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:25.248267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:25.248267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.248267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:25.248267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.248267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:25.708265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:25.708843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:25.709042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:25.710208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.710611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.710966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.711960 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.713411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.714075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:25.714612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.714612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:25.715206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:25.716033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:25.716962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:25.717487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.717630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.718813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:25.720129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.720603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.721417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:25.721814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:25.722180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.722180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:25.722180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.722180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:25.723082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.723538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:25.724082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:25.724426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:25.724619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.724925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.725521 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:25.726698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.727150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.728127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:25.728494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:25.728892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.728892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:25.728892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.728892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.729359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:25.729890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:25.730522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:25.730964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.730964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:25.730964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.731448 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.732060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.732434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:25.732814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.732814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:25.733161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:25.733618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:25.734343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:25.734343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:25.734343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:30.199172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:30.199172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:30.199172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:30.199928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:30.236647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.237024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.237884 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.239140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.239886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:30.240581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:30.241316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.241817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:30.241817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:30.241817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.241817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:30.241817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.241817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:30.274574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:30.275066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:30.275277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:30.284644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 01:35:30.284949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=3, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 01:35:30.285060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.320236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.320818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.321606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.323462 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.324061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.324554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.325195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.325195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.325195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.325195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.358870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.359309 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.359901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.360078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.360989 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.361916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.362419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.363107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.363107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.363107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.363107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.391993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.392395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.392990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.393144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.394063 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.395210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.395746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.396399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.396399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.396399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.396399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.426290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.426750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.427348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.427557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.428542 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.430052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.430693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.431380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.431380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.431380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.431380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 11cded32-5fa8-4334-b438-4cd2f1dbfa1a (type: security_groups) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.485548 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.db.ovn_revision_numbers_db [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Successfully bumped revision number for resource 11cded32-5fa8-4334-b438-4cd2f1dbfa1a (type: security_groups) to 9
May 05 01:35:30.485730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c5644c2c-067f-4b43-bc5a-1bd1d833eac2 (type: security_groups) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.543334 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.db.ovn_revision_numbers_db [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Successfully bumped revision number for resource c5644c2c-067f-4b43-bc5a-1bd1d833eac2 (type: security_groups) to 15
May 05 01:35:30.543702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 7444d70f-195f-4a2e-b51b-a44ca148d836 (type: security_groups) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.584369 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.db.ovn_revision_numbers_db [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Successfully bumped revision number for resource 7444d70f-195f-4a2e-b51b-a44ca148d836 (type: security_groups) to 10
May 05 01:35:30.584369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.704041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.704409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.704988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.705299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.705572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.706647 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.707040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 01:35:30.709099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.709788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.710320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:35:30.710684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:30.721625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:30.725776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:30.735340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:30.903802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.905972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.907262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.908132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.908959 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.910439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.911089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.911631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.911631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.911631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.911631 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.635 seconds
May 05 01:35:30.911631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:30.911631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:30.911951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.912782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.912782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.912910 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.913951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.914486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.915111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.915111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:30.915235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:30.917343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.918256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.918750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:30.920467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.920938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.922883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:30.923374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:30.923891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:30.924360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:30.924691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.925198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:30.925703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.926087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:30.926431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:30.926833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:30.927442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.927804 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.928890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.929213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:30.929519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:30.929969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:30.930510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:30.930510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:30.930510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:30.930510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:30.930510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:30.930510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:35.198373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:35.199299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:35.199471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:35.200583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:35.238018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.238404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.239355 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.240407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:35.241712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:35.242169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.242907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:35.242907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:35.242907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.242907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:35.242907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.242907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:35.911285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:35.912379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:35.912655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:35.914025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.914373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.914610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.915700 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:35.916242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.917910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.918503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:35.919104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.919104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:35.919881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:35.920370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:35.920893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:35.921120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.921413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.922208 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:35.922960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.923438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.923807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:35.924175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:35.924546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.924546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:35.924546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.924546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:35.924780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.925244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:35.925753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:35.926079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.926079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.926565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.927182 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:35.928138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.928573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.929119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:35.929467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:35.929855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.929855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:35.929855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.929855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.930444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:35.930955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:35.931515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.931773 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:35.931773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.932229 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.933134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.933502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:35.933857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.933857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:35.934161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:35.934613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:35.935151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:35.935151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:35.935151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:40.199423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:40.200568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:40.200827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:40.202219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:40.242197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.242528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.243453 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.244299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.244928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:40.245537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:40.246246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.247455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:40.247455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:40.247455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.247455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:40.247455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.247455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:40.912362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:40.912975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:40.913182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:40.914195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.914518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.914885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.916505 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.919329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.919984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:40.920563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:40.921246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:40.922678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.923201 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:40.923667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.924118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.924371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:40.924747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:40.925168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.925168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:40.925168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.925168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:40.925284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.925729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:40.926348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:40.926579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:40.926698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:40.926837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:40.927164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.927383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.927796 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:40.928501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.928915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.929311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:40.929645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:40.930033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.930033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:40.930033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.930033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.930371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:40.930906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:40.931453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:40.931706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.931887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:40.932196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.933016 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.933901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.934196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:40.934453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.934453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:40.934907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:40.935249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:40.935612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:40.935612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:40.935612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:45.200575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:45.201477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:45.201669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:45.203187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:45.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.244495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.245780 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.246864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.247472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:45.248361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:45.249127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.249885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:45.249885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:45.249885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.249885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:45.249885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.249885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:45.913215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:45.913795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:45.914018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:45.915009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.915352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.915593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.916474 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.918217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.918752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:45.919093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.919093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:45.919784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:45.920175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.920719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:45.920719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.920719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:45.921168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:45.921513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:45.921691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:45.921909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:45.922359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.922655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.923401 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:45.924593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.924999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.925689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:45.926030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:45.926352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.926352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:45.926352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.926352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:45.926700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.927110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:45.927532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:45.927744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:45.928117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.928515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.929186 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:45.929924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.930301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.930831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:45.931124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:45.931470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.931470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:45.931470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.931470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.932166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:45.932658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:45.933140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:45.933361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.933731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:45.933995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.934710 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.935530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:45.936171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.936171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:45.936506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:45.936918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:45.937387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:45.937387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:45.937387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:50.205748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:50.206815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:50.206815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:50.207900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:50.238911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.239203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.239850 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.240637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.241610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:50.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:50.243159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.244364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:50.244364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:50.244364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.244364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:50.244364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.244364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:50.913855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:50.914587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:50.914697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:50.915936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.916327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.916573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.917524 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.918515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.919214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:50.919800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:50.920470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.921310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:50.921310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:50.921310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.921310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:50.921310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.921310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:50.921680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:50.922033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:50.922172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:50.922321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:50.922729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.922940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.923476 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:50.924571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.925259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.925646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:50.926197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:50.926738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.926738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:50.926738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.926738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:50.928148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.928635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:50.929130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:50.929345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:50.929384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:50.929573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:50.929983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.930304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.931036 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:50.931962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.932376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.933325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:50.933668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:50.934079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.934079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:50.934079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.934079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:50.934079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.934718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:50.935204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:50.935726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.935995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:50.935995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.936653 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.938746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:50.939098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.939098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:50.940146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:50.940615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:50.941133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:50.941133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:50.941133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:55.210250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:55.211047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:55.211047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:55.211724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:55.245363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.245546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.246176 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.247330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.248011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:55.248637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:55.249338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.250220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:55.250220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:55.250220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.250220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:55.250220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.250220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:55.914563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:55.915096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:55.915349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:55.916587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.916989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.917252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.918199 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.919751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.920323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:55.920982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.920982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:55.921176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:55.922036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:35:55.923113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.923583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.924276 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:55.925037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.925747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.926225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:55.926838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:55.927497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.928257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.928913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:55.929300 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:55.929574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:55.929698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:55.929848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:55.930182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.930425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.930923 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:35:55.931654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.931976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.932575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:55.932833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:55.933102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.933102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:55.933102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.933102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.933711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:55.934043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:55.934397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.934397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:55.934397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.934397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:35:55.935086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:35:55.935336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:35:55.935450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:35:55.935603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:35:55.936144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.936498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:35:55.936755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.937454 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.938264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.938601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:35:55.938962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.938962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:35:55.939219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:35:55.939649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:35:55.940248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:35:55.940248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:35:55.940248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:00.211069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:00.211879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:00.211879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:00.212356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:00.252911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.252911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.253190 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.254247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.254619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:00.255003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.255003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:00.255402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:00.255847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.256331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:00.256331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.256331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:00.915181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:00.915630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:00.915896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:00.916909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.917205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.917412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.918558 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.920030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.920953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:00.921521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:00.922365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:00.923259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:00.923822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.923822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.924379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:00.925207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.925686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.926108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:00.926492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:00.926917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.926917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:00.926917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.926917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:00.927296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.927833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.928400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:00.929676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:00.929931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:00.930089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:00.930204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:00.930482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.930704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.931137 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:00.932037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.932579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.932984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:00.933347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:00.933833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.933833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:00.933833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.933833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.934169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:00.934801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:00.935441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.935441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:00.935441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.935441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:00.935441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:00.935671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:00.936054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:00.936266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:00.936597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.937151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:00.937465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.938355 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.940321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.940814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:00.941248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.941248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:00.941419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:00.942105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:00.942841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:00.942841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:00.942841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:05.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:05.212504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:05.212673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:05.213996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:05.247201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.247631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.248318 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.249002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.249464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:05.249898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:05.250379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.250976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:05.250976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:05.250976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.250976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:05.250976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.250976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:05.915667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:05.916664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:05.916963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:05.918385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.918835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.919103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.920080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.921900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.922688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:05.923284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.923284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:05.923680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:05.924481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:05.925639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:05.926063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.926140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.926962 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:05.927731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:05.928563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.929423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.930125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:05.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.931048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:05.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:05.931898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:05.931898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.931898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.932444 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:05.933243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.933732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.934310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:05.934824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:05.935262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:05.935748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:05.936329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:05.936916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:05.936916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:05.936916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.937378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:05.937617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.938356 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.938992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.939265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:05.939500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.939500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:05.939662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:05.940014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:05.940374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:05.940374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:05.940374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:10.212914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:10.213738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:10.213738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:10.214567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:10.249379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.249854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.250968 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.252033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.252807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:10.253613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:10.254623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:10.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:10.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:10.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:10.916664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:10.917289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:10.917491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:10.918725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.919127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.919377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.920340 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.922216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.922855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:10.923450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.923450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:10.924219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:10.925224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:10.926417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.926860 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:10.927620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.928252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.928834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:10.929179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:10.929542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.929542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:10.929542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.929542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:10.930152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.930566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:10.931055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:10.931455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:10.931455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:10.932053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.932467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.933248 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:10.934044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.934438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.935101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:10.935389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:10.935870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.935870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:10.935870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.935870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.936477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:10.936928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:10.937411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:10.937740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.938136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:10.938369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.939090 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.940040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.940383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:10.940891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.940891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:10.941285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:10.941719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:10.942227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:10.942227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:10.942227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:15.213478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:15.214459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:15.214459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:15.215754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:15.256455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.256588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.257066 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.257708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.258334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:15.259445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:15.260918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.262423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:15.262423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:15.262423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.262423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:15.262423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.262423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:15.917666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:15.918132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:15.918545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:15.919524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.919823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.920069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.921006 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.922468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.923106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:15.923646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.923646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:15.924201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:15.925077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:15.925959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.926395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.926622 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:15.927391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:15.928314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.929189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.929872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:15.930437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.931239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:15.932265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:15.932872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:15.932872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.932872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.933289 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:15.934071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:15.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:15.935686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:15.936327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:15.936959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.937459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:15.937861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:15.937861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:15.937861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:15.937861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:15.937977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.938285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:15.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.939269 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.940148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.940414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:15.940662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.940662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:15.941288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:15.941604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:15.941988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:15.941988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:15.941988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:20.214499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:20.215534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:20.215534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:20.216691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:20.219851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=488,neutron:UPDATE=3 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:36:20.254507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.254960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.255924 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.256830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.257474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:20.258122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:20.259003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.260255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:20.260255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:20.260255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.260255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:20.260255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.260255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:20.918311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:20.919202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:20.919409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:20.920713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.921143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.921387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.922320 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.924076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.924969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:20.925549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.925549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:20.925658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:20.926364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:20.927228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:20.927586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.927938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.928653 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:20.929977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.930692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.931424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:20.932083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:20.932711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:20.934396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:20.935242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:20.935242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:20.935242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.935242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.935416 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:20.936187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.936658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.937110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:20.937463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:20.937936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.937936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:20.937936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.937936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.938436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:20.939180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:20.939797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:20.940072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.940338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:20.940579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.941338 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.942262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.942693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:20.943178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.943178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:20.943338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:20.943867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:20.944466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:20.944466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:20.944466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:25.214707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:25.215630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:25.215630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:25.216877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:25.253186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.253608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.255305 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.256721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.257370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:25.257990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.257990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:25.258336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:25.258899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.259471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:25.259471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.259471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:25.919083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:25.919648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:25.919872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:25.920785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.921125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.921411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.922339 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.923414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.923897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:25.924424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.924424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:25.924591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:25.925391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.926280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:25.926280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.926280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:25.926605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:25.926904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:25.927022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:25.927160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:25.927641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.927994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.928725 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:25.929586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.930249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.930652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:25.931179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:25.931733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.932317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.933095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:25.933607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:25.933776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:25.933906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:25.934052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:25.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.934973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.935715 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:25.936603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.937211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.937594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:25.938091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:25.938640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.938640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:25.938640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.938640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.938822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:25.939516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:25.940140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:25.940425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.940425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:25.940575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.941143 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.942043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.942413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:25.942851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.942851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:25.943111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:25.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:25.944192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:25.944192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:25.944192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:30.215519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:30.216447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:30.216447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:30.217636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:30.249109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.249386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.249987 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.250786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.251128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:30.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:30.251732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:30.252204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.252673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:30.252673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.252673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:30.919850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:30.920649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:30.921038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:30.922039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.922438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.922740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.923578 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.925094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.925715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:30.926440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:30.927204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:30.928086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:30.928541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.928541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.929241 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:30.930058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.930728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.931138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:30.931673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:30.932360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.932840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.933353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:30.933915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:30.934234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:30.934351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:30.934475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:30.934768 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.935007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.935446 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:30.936235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.936631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.937192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:30.937610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:30.938003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.938003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:30.938003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.938003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.938297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:30.938748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:30.939304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.939304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:30.939304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.939304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:30.939718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:30.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:30.940212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:30.940400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:30.940892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.941223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:30.941433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.942173 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.943016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.943358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:30.943699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.943699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:30.944067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:30.944480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:30.945006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:30.945006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:30.945006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:35.216257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:35.217286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:35.217286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:35.217644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:35.252423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.252860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.253934 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.255196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.255921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:35.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:35.256771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:35.257299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.257913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:35.257913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.257913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:35.920559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:35.921222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:35.921430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:35.922632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.923002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.923216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.924071 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.925904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.926465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:35.927009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.927009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:35.927580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:35.928261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:35.929071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:35.929412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.929498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.930100 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:35.931322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:35.932183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.932609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.933054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:35.933426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.933957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:35.934542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:35.934803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:35.934850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:35.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:35.935452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.935779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.936483 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:35.937259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.937680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.938215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:35.938563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:35.938987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.938987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:35.938987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.938987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.939400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:35.939922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:35.940468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.940468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:35.940468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.940468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:35.940468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:35.940468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:35.940670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:35.941043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:35.941106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.941317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:35.941466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.942012 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.943085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.943469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:35.943889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.943889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:35.944117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:35.944584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:35.945195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:35.945195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:35.945195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:40.217005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:40.217689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:40.217689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:40.218205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:40.249432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.250022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.251062 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.251974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.252530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:40.253176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:40.253810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.254610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:40.254610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:40.254610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.254610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:40.254610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.254610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:40.921175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:40.921815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:40.921985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:40.923404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.923726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.924040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.925213 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.926561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.927916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:40.928821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:40.929740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:40.932093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:40.933170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.934168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:40.934990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:40.935451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.935785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.936117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:40.936613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:40.936929 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:40.937144 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.937144 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:40.937144 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.937144 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:40.937290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.937628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.938129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:40.938638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:40.939229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:40.939533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.939912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:40.940607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:40.940883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:40.941141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:40.941141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:40.941591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.941983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:40.942202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.942897 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.943696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.944228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:40.944750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:40.945157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:40.945501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:40.945501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:40.945501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:40.945501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:40.945501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:40.945501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:45.217807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:45.218492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:45.218614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:45.219523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:45.259864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.260174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.260830 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.261584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.262147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:45.262653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:45.263320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.264219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:45.264219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:45.264219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.264219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:45.264219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.264219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:45.921962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:45.923115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:45.923218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:45.924093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.924413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.924655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.925452 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.926232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.927261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:45.928264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:45.928862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:45.929605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:45.930833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:45.930833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:45.930833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:45.931308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.931613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.932425 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:45.933284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:45.934197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.934742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.935062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:45.935488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.935925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:45.936382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:45.936652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:45.936652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.936715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.937418 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:45.938290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.938883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.939110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:45.939403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:45.939826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.939826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:45.939826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.939826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.939949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:45.940433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:45.940935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.940935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:45.940935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.940935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:45.941046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:45.941319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:45.941407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:45.941515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:45.942120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.942418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:45.942707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.943565 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.944380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.944994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:45.945593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:45.946368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:45.947252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:45.947252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:45.947252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:45.947252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:45.947252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:45.947252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:50.218590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:50.219522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:50.219522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:50.219976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:50.258303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.258682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.259616 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.260732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.261427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:50.262219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:50.263011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.264177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:50.264177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:50.264177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.264177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:50.264177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.264177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:50.923391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:50.924082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:50.924337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:50.925556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.925964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.926256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.927182 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.929000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.929903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:50.930751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.930751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:50.931484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:50.932507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:50.933674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:50.934073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.934073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.934903 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:50.935862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.936513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.937163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:50.937635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:50.938257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.938257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:50.938257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.938257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:50.938257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:50.938941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.939666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:50.940242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:50.940537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.940537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.940979 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:50.941994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:50.942631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.943494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:50.944042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:50.944615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.944615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:50.944615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.944615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.945000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:50.945754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.946675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:50.946675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.947056 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.948570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:50.949117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.949117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:50.949365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:50.949884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:50.950413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:50.950413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:50.950413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:55.219214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:55.220038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:55.220038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:55.221614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:55.256190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.258098 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.259285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:55.260014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:55.260792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.261731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:55.261731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:55.261731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.261731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:55.261731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.261731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:55.923982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:55.928236 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:55.928940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.928940 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:55.928940 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.928940 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.929294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.930180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:55.931015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:55.932009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:55.933291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:36:55.933597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.933715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.934436 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:55.935369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.935803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.936427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:55.936821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:55.937165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.937165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:55.937165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.937165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:55.937511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.937927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.938391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:55.939173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:55.939457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:55.939602 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:55.939776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:55.940113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.940326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.941027 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:36:55.942024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.942485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.943213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:55.943674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:55.944227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.944227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:55.944227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.944227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.944638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:55.945342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:36:55.946062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:36:55.946450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.946450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:36:55.946543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.947151 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.948254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.948738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:36:55.949279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.949279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:36:55.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:36:55.950147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:36:55.950878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:36:55.950878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:36:55.950878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:00.219686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:00.220373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:00.220685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:00.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:00.256214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.256531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.257455 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.258275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.258822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:00.259256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:00.259843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.260449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:00.260449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:00.260449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.260449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:00.260449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.260449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:00.926178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:00.926544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:00.926626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:00.927713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.928228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.928519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.929367 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.930395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.931115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:00.931816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:00.932693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:00.934118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:00.934727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.934727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.935114 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:00.935901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.936404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.936714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:00.937127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:00.937590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.937590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:00.937590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.937590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:00.937973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.938474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.939032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:00.939489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:00.939732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:00.939856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:00.939995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:00.940363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.940653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.941223 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:00.942052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.942472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.942947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:00.943289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:00.943694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.943694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:00.943694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.943694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.944108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:00.944606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:00.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:00.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:00.945607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:00.945861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:00.945976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:00.946128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:00.946483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.946746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:00.946968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.947498 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.948425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.948783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:00.949148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.949148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:00.949457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:00.950020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:00.950533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:00.950533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:00.950533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:05.220452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:05.221329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:05.221329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:05.222718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:05.262370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.262736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.263722 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.264917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.265324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:05.265688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.265688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:05.266041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:05.266513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.267061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:05.267061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.267061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:05.926973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:05.927481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:05.927555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:05.928556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.928950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.929228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.930221 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.931004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.931518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:05.932254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:05.932913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:05.933706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:05.934152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:05.934299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.934594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.935272 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:05.935790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:05.936485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.937161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.937723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:05.938350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.939118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.939633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:05.939983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:05.940221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:05.940292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:05.940380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:05.940847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.941190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.941964 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:05.942489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.942952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.943235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:05.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:05.943976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:05.944417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.944972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:05.946068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:05.946328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:05.946399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:05.946509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:05.946957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.947323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:05.947506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.948134 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.948712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.949094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:05.949455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:05.949911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:05.950479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:05.950479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:05.950479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:05.950479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:05.950479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:05.950479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:10.221435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:10.222494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:10.222494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:10.223851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:10.255832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.256328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.257249 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.258040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.259048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:10.259991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:10.261202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.262542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:10.262542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:10.262542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.262542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:10.262542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.262542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:10.927732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:10.928455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:10.928693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:10.930096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.930463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.930754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.931619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.935218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.935905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:10.936897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:10.937935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:10.938818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.939337 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:10.939957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.940636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.941076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:10.941396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:10.941751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.941751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:10.941751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.941751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:10.942336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.942784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:10.943240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:10.943515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:10.943859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.944222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.945168 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:10.945724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.946168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.946407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:10.946780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:10.947155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.947155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:10.947155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.947155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:10.947756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:10.948505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:10.948745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.948980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:10.949204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.949868 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.950510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.950857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:10.951194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.951194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:10.951251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:10.951645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:10.952168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:10.952168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:10.952168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:15.222164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:15.223002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:15.223122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:15.224580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:15.259575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.260008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.261053 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.262254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.262852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:15.263407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:15.264137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.264955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:15.264955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:15.264955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.264955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:15.264955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.264955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:15.928508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:15.929068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:15.929157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:15.930668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.931320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.931543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.932911 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.934729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.935579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:15.936363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.936363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:15.936939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:15.937994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:15.939259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.939831 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:15.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:15.941195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.941728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.942230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:15.942679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.943703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:15.944385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:15.944838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:15.944838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.944838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.944993 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:15.945538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.946091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.946456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:15.946882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:15.947390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:15.947923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:15.948669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:15.949098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:15.949098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:15.949874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.950369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:15.950629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.951566 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.952397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.952816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:15.953159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.953159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:15.953400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:15.953861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:15.954369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:15.954369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:15.954369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:20.222973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:20.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:20.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:20.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:20.232102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=241 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:37:20.253352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.253674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.254943 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.255669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.256000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:20.256256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.256256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:20.257152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:20.257505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.257912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:20.257912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.257912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:20.929096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:20.929776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:20.930080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:20.931274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.931653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.932216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.933035 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.934126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.934516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:20.934872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.934872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:20.935427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:20.935875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.936330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:20.936330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.936330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:20.938224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:20.938690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:20.938842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:20.939015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:20.939699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.939699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.939923 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:20.940936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.941249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.942014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:20.942258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:20.942496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.942496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:20.942496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.942496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:20.943730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.944406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:20.944910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:20.945107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:20.945870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.945870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.946068 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:20.947129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.948042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.948445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:20.948797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:20.949131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.949131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:20.949131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.949131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:20.949131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.949747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:20.950218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:20.950658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.952169 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.953059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.953713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:20.954433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:20.955010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:20.955469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:20.955469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:20.955469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:20.955469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:20.955469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:20.955469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:25.224678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:25.225692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:25.225692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:25.226660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:25.265101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.265491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.266418 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.267494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.267907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:25.268408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.268408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:25.269441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:25.269944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.270443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:25.270443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.270443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:25.929902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:25.930546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:25.930730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:25.932147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.932590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.932852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.933805 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.935477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.936118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:25.936677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.936677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:25.937254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:25.938327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.939224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:25.939224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.939224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:25.939224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:25.939224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:25.939420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:25.939460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:25.940053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.940291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.940977 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:25.941914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.942408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.942871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:25.943209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:25.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:25.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.943562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:25.944095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.944542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:25.945003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:25.945272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:25.945548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.945962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.946834 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:25.947736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.948204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.948746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:25.949106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:25.949481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.949481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:25.949481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.949481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.950238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:25.950842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:25.951329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:25.952259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.952650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:25.952932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.953748 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.954825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.955256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:25.955590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.955590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:25.955926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:25.956397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:25.956941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:25.956941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:25.956941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:30.225198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:30.226062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:30.226062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:30.227426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:30.263728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.264225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.265133 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.266400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:30.268144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:30.268786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.269314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:30.269314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:30.269314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.269314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:30.269314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.269314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:30.930485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:30.931189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:30.931421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:30.933046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.933471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.933796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.934787 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.936540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.936961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:30.937308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.937308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:30.938505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:30.939354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:30.940021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:30.940544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.941061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.941818 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:30.942652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.943247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.943506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:30.944315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:30.944686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.945111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:30.945643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:30.946039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:30.946039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:30.946039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.946309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.947087 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:30.947906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.948332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.948849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:30.949179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:30.949537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.949537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:30.949537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.949537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.949975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:30.950451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:30.950992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.950992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:30.950992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.950992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:30.950992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:30.951283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:30.951410 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:30.951566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:30.952176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.952565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:30.952944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.953792 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.954656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.955040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:30.955389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.955389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:30.955633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:30.956108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:30.956628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:30.956628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:30.956628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:35.225920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:35.226718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:35.227167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:35.228346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:35.263627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.264027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.264952 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.265985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.266671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:35.267415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:35.268221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.269927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:35.269927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:35.269927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.269927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:35.269927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.269927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:35.931243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:35.931893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:35.932109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:35.933545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.933970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.934214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.935281 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.936985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.937306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:35.937626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.937626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:35.939037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:35.939444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.939948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:35.939948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.939948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:35.940036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:35.940284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:35.940432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:35.940593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:35.941142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.941466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.942435 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:35.943372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.943779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.944436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:35.944753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:35.945088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.945088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:35.945088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.945088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:35.945775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.946148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:35.946562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:35.946840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:35.947243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.947683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.948530 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:35.949458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.949807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.950568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:35.950859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:35.951195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.951195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:35.951195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.951195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.952214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:35.952614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:35.953047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:35.953465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.953692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:35.953870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.954399 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.955445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.955739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:35.956069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.956069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:35.956697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:35.957081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:35.957523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:35.957523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:35.957523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:40.226657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:40.227294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:40.227294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:40.228557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:40.262582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.263044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.263980 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.264913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.265754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:40.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:40.267527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.269699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:40.269699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:40.269699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.269699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:40.269699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.269699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:40.931938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:40.932511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:40.932652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:40.933943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.934275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.934535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.935367 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.936371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.936866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:40.937273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.937273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:40.937508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:40.938008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.938659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:40.938659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.938659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:40.940428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:40.940787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:40.940959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:40.941122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:40.941530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.941721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.942277 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:40.943187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:40.944370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.945061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.945715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:40.946327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.947555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:40.948611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:40.949174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:40.949861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:40.950708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:40.951595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:40.952368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.953173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.954302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:40.955522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:40.955522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:40.955522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:40.955522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:40.955522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.955522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.956093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.957148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:40.957860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:40.958389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:40.959561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:45.227339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:45.228062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:45.228335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:45.229695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:45.263648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.264073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.264933 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.265751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.266028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:45.266435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.266435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:45.266592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:45.266956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.267327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:45.267327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.267327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:45.932875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:45.933355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:45.933569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:45.934805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.935150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.935467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.936472 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.938411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.938835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:45.939097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.939097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:45.940092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:45.940431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.940852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:45.940852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.940852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:45.940961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:45.941183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:45.941291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:45.941411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:45.941816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.942101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.942662 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:45.943514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.943892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.944337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:45.944592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:45.944849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.944849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:45.944849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.944849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:45.945348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.945674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.946351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:45.947111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:45.947379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:45.947467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:45.947602 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:45.947975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.948222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.948781 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:45.949476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.950101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.950427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:45.950793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:45.951171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.951171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:45.951171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.951171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.951388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:45.951950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:45.952455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.952455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:45.952455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.952455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:45.953241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:45.953549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:45.953662 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:45.953863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:45.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.954466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:45.954608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.955092 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.955902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.956262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:45.956615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.956615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:45.956841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:45.957371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:45.957905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:45.957905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:45.957905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:50.228067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:50.228971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:50.228971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:50.230050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:50.261213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.261627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.262545 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.263312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.263942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:50.264569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:50.265404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.266782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:50.266782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:50.266782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.266782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:50.266782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.266782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:50.933368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:50.933887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:50.934058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:50.935116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.935364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.935520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.936203 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.937703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.938334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:50.938947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.938947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:50.939479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:50.940231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.941109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:50.941109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.941109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:50.941360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:50.941742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:50.942072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:50.942280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:50.942627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.942883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.943579 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:50.944970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.945419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.946419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:50.946803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:50.947162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.947162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:50.947162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.947162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:50.947596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.948031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:50.948511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:50.948814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:50.948814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.949110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.949936 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:50.950711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.951123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.951655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:50.952005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:50.952411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.952411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:50.952411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.952411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.952739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:50.953309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:50.953845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.953845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:50.953845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.953845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:50.953845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:50.953845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:50.954044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:50.954082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:50.954525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.954974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:50.955177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.955842 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.956648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.956990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:50.957330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.957330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:50.957720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:50.958169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:50.958647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:50.958647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:50.958647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:55.228824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:55.229755 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:55.229755 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:55.231044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:55.266060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.266594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.267787 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.269739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.270351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:55.270921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.270921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:55.271484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:55.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:55.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:55.934110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:55.934795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:55.935019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:55.936056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.936369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.936608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.937530 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.939234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.939592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:55.939955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.939955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:55.941098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:55.941539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.942061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:55.942061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.942061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:55.942061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:55.942061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:55.942210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:55.942340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:37:55.942776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.943136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.943858 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:55.944588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.945053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.945431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:55.945804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:55.946139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.946139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:55.946139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.946139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:55.946481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.946932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.947448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:55.948180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:55.948523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:55.948642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:55.948780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:55.949214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.949567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.950473 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:37:55.951301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.952156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:55.952489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:55.952878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.952878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:55.952878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.952878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.953218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:55.953773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:37:55.954307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:37:55.954514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:37:55.954775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.955038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:37:55.955177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.955686 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.956503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.956894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:37:55.957256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.957256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:37:55.957502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:37:55.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:37:55.958615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:37:55.958615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:37:55.958615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:00.229548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:00.230498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:00.230606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:00.232781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:00.270005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.270406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.271294 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.272427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.273166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:00.273839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.273839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:00.274183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:00.274725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.275256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:00.275256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.275256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:00.934874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:00.935546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:00.935751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:00.936756 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.937028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.937164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.937932 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.939458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.940300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:00.941164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.941164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:00.941383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:00.942333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:00.943631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.944307 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:00.945074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:00.946161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.946703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.947199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:00.947864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.948351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:00.948816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:00.949018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:00.949018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:00.949166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:00.949443 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.949615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.950619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:00.951360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.951752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.952224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:00.952542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:00.952900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.952900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:00.952900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.952900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.953346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:00.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:00.954262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.954262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:00.954262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.954262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:00.954367 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:00.954567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:00.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:00.954863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:00.955111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.955276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:00.955382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.955792 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.956631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.957130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:00.957618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.957618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:00.958136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:00.958842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:00.959495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:00.959495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:00.959495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:05.230214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:05.230762 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:05.230962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:05.232278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:05.266725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.267280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.268353 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.269203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.270022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:05.270713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:05.271571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.272413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:05.272413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:05.272413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.272413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:05.272413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.272413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:05.935845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:05.936405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:05.936664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:05.937919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.938304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.938539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.939538 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.940805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.941503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:05.942383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:05.943269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:05.944551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.944908 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:05.945232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.945665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.945911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:05.946252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:05.946692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.946692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:05.946692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.946692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:05.946940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.947434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.948024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:05.949135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:05.949528 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:05.949733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:05.950017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:05.950774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.951257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.952305 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:05.954521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:05.955036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.956560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:05.956920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:05.957320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.957320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:05.957320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.957320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.958500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:05.959003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:05.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:05.959721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.959966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:05.960114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.960770 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.961708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.962125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:05.962542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:05.963007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:05.963564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:05.963564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:05.963564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:05.963564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:05.963564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:05.963564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:10.231057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:10.232003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:10.232003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:10.232288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:10.266974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.267203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.267984 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.268788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.269635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:10.270459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:10.271233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.272153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:10.272153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:10.272153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.272153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:10.272153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.272153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:10.936587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:10.937095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:10.937229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:10.937983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.938278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.938526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.939404 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.940369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.940921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:10.941310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.941310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:10.942488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:10.943028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:10.943695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:10.944126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.944476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.945438 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:10.946048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.946566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.946881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:10.947230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:10.947557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.947557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:10.947557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.947557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:10.948955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.949329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:10.949659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:10.949826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:10.949919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:10.950013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:10.950425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.950785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.951400 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:10.951920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.952253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.952561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:10.952810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:10.953079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.953079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:10.953079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.953079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.953308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:10.953675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:10.954076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.954076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:10.954076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.954076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:10.959102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:10.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:10.959479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:10.959605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:10.960018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.960405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:10.960605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.961240 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.961862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.962256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:10.962586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:10.963017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:10.963524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:10.963524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:10.963524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:10.963524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:10.963524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:10.963524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:15.231631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:15.232556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:15.232805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:15.234444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:15.266311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.266622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.267281 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.268050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.268507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:15.268950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:15.269460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.270122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:15.270122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:15.270122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.270122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:15.270122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.270122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:15.937583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:15.938337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:15.938597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:15.939864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.940238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.940514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.941499 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.943201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.943715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:15.944279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.944279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:15.944935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:15.945722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:15.946644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.947018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.947703 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:15.948423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:15.949641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.950415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.951320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:15.952472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.953223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:15.953681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:15.953953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:15.954255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:15.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:15.954981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:15.955285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.955607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.956097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:15.959566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:15.959965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:15.960080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:15.960237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:15.960674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.961054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:15.961413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.963978 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.965018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.965301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:15.965554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.965554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:15.966148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:15.967492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:15.968265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:15.968265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:15.968265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:20.232383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:20.233004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:20.233004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:20.233529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:20.239189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:20.277812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.279417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.279792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.280185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:20.938354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:20.938924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:20.939198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:20.940626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.941099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.941420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.942646 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.944628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.945085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:20.945483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.945483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:20.946919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:20.947430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:20.948048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:20.948326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.948452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.949004 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:20.949958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:20.951331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.952484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.953652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:20.954643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.955934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:20.957621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:20.958690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:20.960074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:20.961244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:20.962306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:20.962843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.963146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.963627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:20.964132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:20.964132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:20.964132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:20.964132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:20.964132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.964132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.964337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.964714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.965140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:20.965450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:20.965945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:20.966448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:20.966448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:20.966448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:25.233055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:25.233926 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:25.234205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:25.235403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:25.268000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.268522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.269498 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.270785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.271550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:25.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:25.272841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.273566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:25.273566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:25.273566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.273566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:25.273566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.273566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:25.939090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:25.939644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:25.939879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:25.941113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.941536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.941958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.942955 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.944425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.945427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:25.946074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.946074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:25.946409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:25.947559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:25.948372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:25.948810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.949054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:25.950324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.950799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.951423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:25.953646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:25.954637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.955385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:25.955974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.956258 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:25.956489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.956743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:25.958300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:25.958981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:25.959236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.959553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:25.960683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:25.960946 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:25.961056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:25.961212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:25.961583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.961848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:25.962036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.962525 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.963359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.964818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:25.965143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:25.965444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:25.965912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:25.965912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:25.965912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:25.965912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:25.965912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:25.965912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:30.235127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:30.235842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:30.236062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:30.237407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:30.274213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.274519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.275272 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.276902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:30.277316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.277316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:30.277387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:30.277883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.278503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:30.278503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.278503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:30.939746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:30.940508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:30.940798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:30.941967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.942350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.942611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.943555 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.945532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.946516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:30.947382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.947382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:30.947974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:30.948538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:30.949260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:30.949611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.949831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.950338 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:30.951134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.951692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.952237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:30.952989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:30.953482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.954005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:30.954886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:30.955398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:30.955398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:30.955398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.955929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.956824 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:30.957846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.958509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.959247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:30.959711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:30.960239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.960239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:30.960239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.960239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:30.961666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:30.962390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:30.962668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.962836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:30.963227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.964265 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.965398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.965913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:30.966375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.966375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:30.966488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:30.967071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:30.967750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:30.967750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:30.967750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:35.235733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:35.236912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:35.236912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:35.238263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:35.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.275411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.276561 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.277824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.279330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:35.279739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:35.280267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.280913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:35.280913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:35.280913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.280913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:35.280913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.280913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:35.940533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:35.941640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:35.941640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:35.943275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.943572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.943770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.944788 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.946314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.947150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:35.947827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:35.948658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:35.949705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:35.952436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.952643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.953483 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:35.954815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:35.955591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.958173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:35.958574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:35.958968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.958968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:35.958968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.958968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:35.962478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.963066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.963660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:35.964745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:35.965540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:35.965686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:35.966011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:35.966636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.967074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.967546 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:35.968365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.968947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.969231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:35.969581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:35.970003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.970003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:35.970003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.970003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.970136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:35.970672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:35.971368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.971655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:35.971655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.971967 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.972743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.973189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:35.973539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:35.974027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:35.974618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:35.974618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:35.974618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:35.974618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:35.974618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:35.974618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:40.236437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:40.237102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:40.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:40.238658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:40.276826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.276965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.277443 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.278142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.278504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:40.278848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.278848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:40.278908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:40.279319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.279810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:40.279810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.279810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:40.941105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:40.941714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:40.941935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:40.942883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.943134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.943293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.943956 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.945461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.945877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:40.946211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.946211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:40.947178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:40.947618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.948068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:40.948068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.948068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:40.949559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:40.949953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:40.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:40.950314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:40.953236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.953415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:40.954841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.955302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.955931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:40.956246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:40.956594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.956594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:40.956594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.956594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:40.957177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.957612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.958070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:40.965438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:40.965712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:40.965869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:40.966035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:40.966533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.966831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.967382 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:40.968176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.968572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.969322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:40.969578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:40.969920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.969920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:40.969920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.969920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.970716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:40.971128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:40.971535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:40.971717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:40.973124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.973345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:40.973489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.974038 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.974933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.975238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:40.975520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.975520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:40.976095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:40.976527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:40.977136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:40.977136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:40.977136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:45.237113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:45.238025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:45.238025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:45.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:45.272029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.272226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.272937 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.274111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.274804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:45.275443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.275443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:45.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:45.276611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.277582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:45.277582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.277582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:45.941795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:45.942695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:45.942977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:45.944365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.944697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.944979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.945832 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.947504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.947937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:45.948333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.948333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:45.950129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:45.950715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.951286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:45.951286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.951286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:45.951286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:45.951286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:45.951714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:45.951848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:45.952902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.953180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.953710 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:45.955341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.955916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.956524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:45.956984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:45.957409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.957409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:45.957409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.957409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:45.957543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.958112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.960154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:45.965775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:45.966160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:45.966349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:45.966586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:45.967304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.969241 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:45.969837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.970346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.971338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:45.971707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:45.972151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.972151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:45.972151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.972151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:45.972151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.973252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:45.974025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:45.974583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:45.974847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.975195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.975985 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.977500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.977956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:45.978346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.978346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:45.978703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:45.979166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:45.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:45.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:45.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:50.237647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:50.238260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:50.238260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:50.239499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:50.274640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.274931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.275613 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.276285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.276562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:50.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:50.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:50.277429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.277888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:50.277888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.277888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:50.942688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:50.943044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:50.943208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:50.943959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.944197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.944352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.944949 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.945603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.946066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:50.946476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:50.947248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.948227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:50.948227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:50.948227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.948227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:50.948227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.948227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:50.950947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:50.951122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:50.951191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:50.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:50.951612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.951890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.952357 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:50.952888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:50.953502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.954208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.954872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:50.955566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.956269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.957188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:50.966143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:50.966868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:50.966980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:50.967158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:50.968033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.968501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.969565 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:50.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.972189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.974135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:50.974868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.976232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:50.977101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:50.978010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:50.978353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:50.978623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.979296 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.980786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.981393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:50.982133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.982133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:50.982437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:50.983491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:50.984439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:50.984439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:50.984439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:55.238123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:55.239173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:55.239173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:55.240347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:55.284954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.285258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.286063 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.291558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.293352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:55.293912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:55.294402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.295048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:55.295048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:55.295048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.295048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:55.295048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.295048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:55.943378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:55.948587 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:55.950010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:55.950824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.950824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:55.950824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.950824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:55.951951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:55.952704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:38:55.953633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.954080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:55.954830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:55.955597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.956336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.956954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:55.957446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.958114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.958878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:55.967049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:55.967345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:55.967429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:55.967639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:55.968550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.968997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:38:55.971541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.973831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:55.974190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:55.974612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.974612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:55.974612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.974612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.975715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:55.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:55.976923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.976923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:55.976923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.976923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:38:55.977192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:38:55.977423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:38:55.977513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:38:55.977578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:38:55.978035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.978349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:38:55.978509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.979599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.980368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:38:55.980995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:38:55.981834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:38:55.982799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:38:55.982799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:38:55.982799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:38:55.982799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:38:55.982799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:38:55.982799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:00.239189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:00.240049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:00.240049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:00.243804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:00.286126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.286377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.287008 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.288068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.288495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:00.288898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.288898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:00.289267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:00.289792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.290362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:00.290362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.290362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:00.943959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:00.944377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:00.944573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:00.945636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.945988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.946217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.947266 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.948482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.949039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:00.949637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.949637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:00.950100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:00.950777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.951468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:00.951468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.951468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:00.952679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:00.952993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:00.953150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:00.953360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:00.953718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.953938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.954467 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:00.955467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.956107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.956987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:00.957555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:00.958183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.958183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:00.958183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.958183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:00.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.959152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.959771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:00.967462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:00.967857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:00.968026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:00.968216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:00.968700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.969125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.969972 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:00.971116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.972462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:00.972819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:00.973377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.973377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:00.973377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.973377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.975350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:00.975980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:00.976813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.976813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:00.976813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.976813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:00.977662 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:00.978011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:00.978105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:00.978221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:00.978527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.978765 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:00.978935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.979452 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.980099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.980478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:00.980962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:00.981432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:00.982037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:00.982037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:00.982037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:00.982037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:00.982037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:00.982037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:05.240160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:05.241056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:05.241056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:05.242076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:05.274805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.275068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.275704 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.276448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.277078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:05.277480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:05.278018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.278617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:05.278617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:05.278617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.278617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:05.278617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.278617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:05.944538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:05.945696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:05.945696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:05.948044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.948419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.948824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:05.949150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:05.949547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.950031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:05.950031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.950031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:05.953099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:05.953697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:05.953881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:05.954144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:05.954745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.955066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.956965 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:05.957656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.958117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.958667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:05.959029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:05.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:05.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:05.960579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.961011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.961375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:05.968013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:05.968240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:05.968321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:05.968469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:05.969053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.969331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.969907 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:05.970848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.971265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.971791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:05.972534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:05.973443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.973443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:05.973443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.973443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.975236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:05.975793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:05.976238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.976238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:05.976238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.976238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:05.978081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:05.978258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:05.978371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:05.978535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:05.979110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.979650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:05.980150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.980887 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.981775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.982126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:05.982440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.982440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:05.984180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:05.984678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:05.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:05.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:05.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:10.240721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:10.241553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:10.241553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:10.241865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:10.273724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.274075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.275379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.277080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.277771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:10.278429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:10.279139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.279944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:10.279944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:10.279944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.279944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:10.279944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.279944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:10.945165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:10.945605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:10.946267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:10.946694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.947142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.947389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.948282 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.949480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.950177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:10.951464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:10.952225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.953393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:10.953393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:10.953393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.953393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:10.953393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.953393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:10.953819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:10.954116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:10.954212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:10.954333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:10.954935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.955194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.955772 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:10.956532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.957056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.957362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:10.957736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:10.958154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.958154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:10.958154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.958154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:10.958480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.959483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.960060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:10.968578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:10.968890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:10.968890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:10.969087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:10.970071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.970886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.971688 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:10.972533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.973139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.973442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:10.973877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:10.974268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.974268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:10.974268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.974268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:10.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:10.975970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.975970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:10.975970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.975970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:10.978431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:10.978673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:10.978802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:10.979321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:10.979666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.980284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:10.980495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.981318 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.982159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.982585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:10.983044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.983044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:10.983115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:10.983655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:10.984359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:10.984359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:10.984359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:15.241132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:15.241820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:15.241820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:15.242503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:15.281522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.281522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.281766 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.282689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.283260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:15.283871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:15.284582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.285484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:15.285484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:15.285484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.285484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:15.285484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.285484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:15.945830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:15.946530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:15.946730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:15.948933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.948933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.948933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.949437 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.951754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:15.952334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.952334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:15.952851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:15.953558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:15.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:15.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:15.955307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:15.955536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:15.955748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:15.956274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.957386 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:15.958881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.959562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.959945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:15.960475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:15.961190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.961804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.962616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:15.968730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:15.969020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:15.969136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:15.969274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:15.969647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.969945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.970426 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:15.971348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.971827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.972218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:15.972570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:15.972989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.972989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:15.972989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.972989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.973316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:15.973809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:15.974402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.974402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:15.974402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.974402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:15.978993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:15.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:15.979578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:15.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:15.980228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.980533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:15.980668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.981186 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.982004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.982703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:15.983380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:15.984004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:15.984735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:15.984735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:15.984735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:15.984735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:15.984735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:15.984735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:20.242012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:20.242871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:20.242871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:20.244188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:20.250179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:39:20.279564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.279821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.280369 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.281282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.281709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:20.282114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.282114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:20.282349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:20.282820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.283381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:20.283381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.283381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:20.946624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:20.947325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:20.947579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:20.948962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.949337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.949666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.950713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.953070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.954327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:20.955183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.955183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:20.955649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:20.956731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:20.957992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.958523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.959506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.960143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:20.960631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:20.961188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.961808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.962535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:20.969247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:20.969878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:20.970152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:20.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:20.971238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.971453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.972338 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:20.973497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.974485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.974923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:20.975390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:20.975919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.975919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:20.975919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.975919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.976455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:20.977171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:20.977891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.977891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:20.977891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.977891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:20.979694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:20.980028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:20.980092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:20.980252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:20.980746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.981130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:20.981354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.982049 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.982995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:20.984076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:20.984684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:20.985605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:20.985605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:20.985605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:20.985605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:20.985605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:20.985605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:25.242664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:25.243614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:25.243614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:25.244838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:25.277200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.277627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.278516 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.279203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.279932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:25.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:25.281261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.282139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:25.282139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:25.282139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.282139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:25.282139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.282139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:25.947339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:25.948017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:25.948286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:25.949639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.950080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.950397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.951507 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.952960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.953501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:25.954045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.954045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:25.956032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:25.956600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.957104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:25.957104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.957104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:25.957104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:25.957104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:25.957895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:25.958141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:25.958615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.958886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.959488 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:25.960393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.960914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.961254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:25.961779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:25.962262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.962262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:25.962262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.962262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:25.962378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.962886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.963387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:25.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:25.969894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:25.970026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:25.970158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:25.970563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.970885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.971456 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:25.972231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.972626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.973558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:25.974130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:25.974707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.974707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:25.974707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.974707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.976744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:25.977568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:25.978642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.978642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:25.978642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.978642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:25.980196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:25.980448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:25.980538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:25.980713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:25.981240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.981622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:25.981903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.982700 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.983582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.984219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:25.984835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:25.985659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:25.986686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:25.986686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:25.986686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:25.986686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:25.986686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:25.986686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:30.243390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:30.243936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:30.244135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:30.245642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:30.288568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.288999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.289880 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.291264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.291875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:30.292463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:30.293277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.294191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:30.294191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:30.294191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.294191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:30.294191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.294191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:30.948191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:30.948734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:30.948920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:30.950418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.950856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.951129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.952076 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.953785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.954348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:30.954687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.954687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:30.955211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:30.955662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.956192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:30.956192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.956192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:30.956919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:30.957772 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:30.958026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:30.958148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:30.958661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.958931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.959529 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:30.960306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.960665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.961121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:30.961381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:30.961633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.961633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:30.961633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.961633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:30.962160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.962483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.963005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:30.969949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:30.970157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:30.970253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:30.970389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:30.970863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.971119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.972002 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:30.973249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.974746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:30.975407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:30.976078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.976078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:30.976078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.976078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.976619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:30.977530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:30.978496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.978496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:30.978496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.978496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:30.980492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:30.980672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:30.980783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:30.980981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:30.981600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.982229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:30.982501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.983261 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.984171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.984567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:30.984943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.984943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:30.985200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:30.985697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:30.986495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:30.986495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:30.986495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:35.244296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:35.245009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:35.245142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:35.246829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:35.287619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.288059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.289060 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.290722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:35.291313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:35.292012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.292861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:35.292861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:35.292861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.292861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:35.292861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.292861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:35.948838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:35.949118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:35.949370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:35.950274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.950603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.950875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.951733 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.952696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.953383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:35.954014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:35.954645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.955778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:35.955778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:35.955778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.955778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:35.955778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.955778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:35.957825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:35.958027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:35.958121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:35.958244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:35.958587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.958836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.959342 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:35.960386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.960865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.961450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:35.961928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:35.962328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.962328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:35.962328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.962328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:35.962467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.963037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.963561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:35.970280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:35.970626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:35.970692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:35.970947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:35.971462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.971855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.972625 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:35.974147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.974816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:35.976261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:35.977049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.977049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:35.977049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.977049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.978463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:35.979182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:35.980290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.980290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:35.980290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.980290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:35.980734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:35.980859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:35.980947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:35.981120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:35.981449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.981787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:35.982056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.982952 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.983579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.983947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:35.984378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:35.984859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:35.985439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:35.985439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:35.985439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:35.985439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:35.985439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:35.985439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:40.244900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:40.245794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:40.245794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:40.246197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:40.283440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.284207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.285327 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.286189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.286806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:40.287370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:40.288129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.289052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:40.289052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:40.289052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.289052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:40.289052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.289052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:40.949521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:40.949927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:40.950048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:40.951416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.951965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.952571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.953401 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.954854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.955490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:40.956206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:40.957249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:40.958211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:40.958730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:40.958730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:40.958730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:40.959232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.959607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.960495 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:40.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:40.962371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.963179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.963909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:40.964479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.966774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:40.970709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:40.970996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:40.971087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:40.971239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:40.971937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.972308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.973110 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:40.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.974562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.974948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:40.975329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:40.975717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.975717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:40.975717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.975717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.975860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:40.976340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:40.977007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.977007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:40.977007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.977007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:40.980964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:40.981125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:40.981206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:40.981360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:40.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.982287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:40.982521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.983282 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.983943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.984305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:40.984669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:40.985115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:40.985612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:40.985612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:40.985612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:40.985612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:40.985612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:40.985612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:45.245550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:45.246313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:45.246313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:45.247792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:45.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.287089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.288117 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.289845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:45.290580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:45.291361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:45.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:45.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:45.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:45.950037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:45.950341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:45.950442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:45.951480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.951853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.952168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.953077 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.954132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.954685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:45.955274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:45.955911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.957263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:45.957263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:45.957263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.957263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:45.957263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.957263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:45.958411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:45.958601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:45.958710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:45.958851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:45.959263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.960022 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:45.961046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.961705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.962115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:45.962677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:45.963272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.963913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.964865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:45.971020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:45.971216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:45.971328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:45.971455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:45.971936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.972185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.972699 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:45.973421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.973708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.974304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:45.974825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:45.975443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.975443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:45.975443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.975443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.975664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:45.976418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:45.977283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.977283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:45.977283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.977283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:45.981223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:45.981526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:45.981621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:45.981782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:45.982150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.982380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:45.982530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.983116 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.983927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.984264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:45.984617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.984617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:45.984913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:45.985329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:45.985880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:45.985880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:45.985880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:50.246454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:50.248330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:50.248330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:50.249797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:50.282275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.282541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.283405 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.285564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.286028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:50.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:50.286962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.287508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:50.287508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:50.287508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.287508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:50.287508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.287508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:50.950978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:50.951531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:50.951701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:50.953075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.953471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.953725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.954631 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.955644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.956287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:50.956833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:50.957496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.958443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:50.958443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:50.958443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.958443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:50.958443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.958443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:50.958700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:50.959078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:50.959078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:50.959078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:50.959530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.960032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.960637 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:50.961390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.961954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.962403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:50.963494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:50.964461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.964461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:50.964461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.964461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:50.965165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.966350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.966912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:50.971258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:50.971423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:50.971503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:50.971624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:50.971996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.972214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.972715 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:50.973339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:50.974054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:50.974614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:50.975153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:50.975638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.976398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:50.981556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:50.981811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:50.981930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:50.982085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:50.982653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.983101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:50.983331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.984115 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.985517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.986051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:50.986420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.986420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:50.987236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:50.987749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:50.988342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:50.988342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:50.988342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:55.248354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:55.248649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:55.248649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:55.249518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:55.290349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.291813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.292398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.293022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:55.293550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:55.294222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.295093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:55.295093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:55.295093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.295093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:55.295093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.295093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:55.951588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:55.952016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:55.952090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:55.953351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.954258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.954670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.956587 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.957420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.957728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:55.958005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.958005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:55.958196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:55.958494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.958850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:55.958850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.958850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:55.959381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:55.959556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:55.959692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:55.959729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:39:55.960362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.960639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.961379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:55.961926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:55.962638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.963866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:55.964406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.965716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.966511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:55.971688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:55.971998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:55.973294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:55.973294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:55.973294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.973294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.973528 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:39:55.974727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.975192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.976115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:55.976489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:55.976937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.976937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:55.976937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.976937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.977869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:55.978350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:55.978813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.978813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:55.978813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.978813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:39:55.981929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:39:55.982286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:39:55.982412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:39:55.982563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:39:55.982963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.983213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:39:55.983383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.983983 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.984901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.985280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:39:55.985840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.985840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:39:55.985976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:39:55.986798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:39:55.987304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:39:55.987304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:39:55.987304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:00.248988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:00.249696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:00.249899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:00.250726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.284520 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.284993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.285671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:00.285949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:00.286221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.286553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:00.286553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:00.286553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.286553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:00.286553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.286553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:00.952288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:00.953033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:00.953157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:00.954524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.954788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.954964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.955724 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.957859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.958729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:00.959919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:00.961340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:00.962455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:00.962854 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.962854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.963591 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:00.965542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.966297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.967867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:00.968270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:00.968606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.968606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:00.968606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.968606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:00.970153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.970832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.971188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:00.972088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:00.972489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:00.972489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:00.972700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:00.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.973346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.974009 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:00.975011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:00.975992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:00.976668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:00.977362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:00.979303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.979924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:00.982299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:00.983005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:00.983144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:00.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:00.983780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.984036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:00.984187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.984693 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.985731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.986143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:00.986488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.986488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:00.987174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:00.987885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:00.988596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:00.988596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:00.988596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:05.249592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:05.250267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:05.250267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:05.251220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:05.289092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.289297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.291014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.291315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:05.291640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.291640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:05.292252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:05.292600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.292981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:05.292981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.292981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:05.952988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:05.954874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:05.954874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:05.955273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.955511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.955670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:05.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:05.960424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.960866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.961798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:05.962584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:05.963546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:05.963748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:05.963969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:05.964391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.964572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.967436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:05.967922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.968304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.968544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:05.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:05.969196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.969571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:05.972573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:05.973403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:05.973403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:05.973403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:05.973667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.977403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:05.978144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.978679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.979171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:05.979677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:05.980322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.981304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:05.983064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:05.983311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:05.983529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:05.984006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:05.984605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.984987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:05.985223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.985860 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.987881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:05.988485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:05.988841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:05.989255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:05.989255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:05.989255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:05.989255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:05.989255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:05.989255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:10.250182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:10.250865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:10.250865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:10.251510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:10.280362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.280687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.281426 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.282229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.282676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:10.283112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:10.283605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:10.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:10.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:10.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:10.953813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:10.954493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:10.954713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:10.955867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.956160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.956390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.957124 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.958527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.959071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:10.959430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.959430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:10.960093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:10.960649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.961233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:10.961233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.961233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:10.963559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:10.964038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:10.964038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:10.964197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:10.964875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.965190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.966702 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:10.967908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.968569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.969101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:10.969493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:10.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:10.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:10.970300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.970851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.972029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:10.973099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:10.973355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:10.973477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:10.973619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:10.974131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.974423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.974982 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:10.976573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:10.977143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:10.977592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.978019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.978375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:10.978924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:10.979501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.979501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:10.979501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.979501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:10.983494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:10.983728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:10.983849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:10.984181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:10.984619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.985018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:10.985269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.985901 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.986818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.987277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:10.987686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.987686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:10.988100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:10.988676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:10.989284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:10.989284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:10.989284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:15.250577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:15.251229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:15.251229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:15.252130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:15.287794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.288257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.290851 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.291696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.292148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:15.292436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.292436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:15.294132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:15.294650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.295143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:15.295143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.295143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:15.954442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:15.955005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:15.955177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:15.955986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.956386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.956541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.957376 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.958362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.958962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:15.959531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:15.960201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.961072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:15.961072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:15.961072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.961072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:15.961072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.961072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:15.964103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:15.964387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:15.964470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:15.964626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:15.964994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.965177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.965565 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:15.966295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.966837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.967116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:15.967543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:15.967994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.967994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:15.967994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.967994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:15.968233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.968727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.969411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:15.973611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:15.973934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:15.974097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:15.974290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:15.974608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.974823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.975221 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:15.976286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.976708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.977618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:15.977980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:15.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:15.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.979291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:15.979793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:15.980256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.980256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:15.980256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.980256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:15.983986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:15.984210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:15.984289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:15.984455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:15.984879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.985337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:15.985488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.986122 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.986881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.987364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:15.987792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:15.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:15.989275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:15.989275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:15.989275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:15.989275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:15.989275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:15.989275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:20.251173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:20.251749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:20.251995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:20.253221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:20.256074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=239 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:40:20.287000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.287249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.287778 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.288521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.289225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:20.290036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:20.290937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.292550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:20.292550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:20.292550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.292550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:20.292550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.292550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:20.955108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:20.955635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:20.955824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:20.957165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.957473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.957797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.958606 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.960381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.960901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:20.961365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.961365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:20.962916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:20.963519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.964237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:20.964237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.964237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:20.964931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:20.964931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:20.965169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:20.965169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:20.965833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.966029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.966658 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:20.967610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.968309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.968720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:20.969060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:20.969378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.969378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:20.969378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.969378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:20.970282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.971428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.971955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:20.974052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:20.974441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:20.974600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:20.974835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:20.975293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.975560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.976255 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:20.977159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.977569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.978421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:20.978730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:20.979084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.979084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:20.979084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.979084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.979669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:20.980285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:20.980711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.980711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:20.980711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.980711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:20.984365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:20.984665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:20.984798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:20.984983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:20.985482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.985708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:20.985868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.986716 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.987743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.988133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:20.988471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.988471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:20.988746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:20.989218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:20.990156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:20.990156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:20.990156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:25.251697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:25.252383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:25.252383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:25.252982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:25.285770 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.286144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.287005 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.287825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.288244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:25.288606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.288606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:25.288827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:25.289526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:25.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.290126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:25.955533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:25.956056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:25.956182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:25.957160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.957332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.957432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.957925 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.958940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.959523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:25.960249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:25.960923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.962203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:25.962203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:25.962203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.962203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:25.962203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.962203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:25.964784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:25.968268 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.968922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.969988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.971026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.972289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:25.972833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:25.973382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.974082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:25.977803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:25.977803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:25.977803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:25.977803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:25.977803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.977803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.978131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:25.978804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.979372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.979896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:25.980373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:25.980923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.981935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:25.984717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:25.984991 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:25.985105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:25.985261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:25.985570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.985804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:25.985943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.986397 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.987403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.987733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:25.988086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.988086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:25.988868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:25.989293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:25.989791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:25.989791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:25.989791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:30.252231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:30.252731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:30.253080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:30.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:30.276135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:30.293027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.293860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.294823 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.296650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:30.297408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:30.298659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:30.300468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:30.308590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 01:40:30.308590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 01:40:30.308590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:40:30.338075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.338667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.338877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.339270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.340137 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.341840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.342385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.342831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.342831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.342831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.342831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:40:30.371560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.371831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.372036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.372131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.372782 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.374165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.374679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.375278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.375278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.375278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.375278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:40:30.409122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.409429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.409743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.409877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.410585 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.411601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.412059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.412652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.412652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.412652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.412652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:40:30.444958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.445199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.445572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.445693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.446451 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.448004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.448370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.448829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.448829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.448829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.448829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:40:30.535604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.535847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.536196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.536631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.536839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.537752 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.538005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 01:40:30.539777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.540118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.540474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:40:30.656038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.656334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.656672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.656939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.657599 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.658473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.658857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.659272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.659272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.659272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.659272 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.361 seconds
May 05 01:40:30.956164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:30.956612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:30.956706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:30.957570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.957820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.957994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.958648 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.959384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.959720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.960010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.960010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:30.960444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:30.960991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.961327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:30.961327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.961327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:30.965259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:30.965549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:30.965662 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:30.965820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:30.966260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.966467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.967037 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:30.968486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.969079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.969321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.969624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.969996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:30.970456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.970899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.971369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:30.976969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:30.976969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:30.976969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:30.976969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:30.976969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.976969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.977257 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:30.977688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:30.979094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.979465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.979729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.979952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:30.980320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:30.980698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.980698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:30.980698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.980698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:30.985168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:30.985609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:30.985825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:30.986152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:30.986565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.986835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:30.986979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.987557 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.992570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.993003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:30.993441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.993441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:30.994126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:30.995140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:30.996365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:30.996365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:30.996365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:35.252869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:35.253874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:35.253874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:35.254183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:35.292110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.292633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.294619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.295487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.295770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:35.296053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.296053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:35.298924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:35.299350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.299962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:35.299962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.299962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:35.957380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:35.957694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:35.957694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:35.959054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.959371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.959543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.960329 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.961647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.962067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:35.962473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.962473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:35.962869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:35.963412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.963952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:35.963952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.963952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:35.965647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:35.965987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:35.966245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:35.966245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:35.967124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.967124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.967899 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:35.968709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.969391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.970032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:35.970418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:35.970819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.970819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:35.970819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.970819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:35.971777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.972273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.972736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:35.975933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:35.975933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:35.975933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:35.975933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:35.976285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.976426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.977031 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:35.977816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:35.978631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:35.979498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:35.980305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:35.981005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.981615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:35.986188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:35.986596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:35.986790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:35.987060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:35.988279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.988570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:35.988738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.989685 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.991028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.991482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:35.991902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.991902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:35.992711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:35.993318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:35.993837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:35.993837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:35.993837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:40.253374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:40.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:40.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:40.255017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:40.290932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.290932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.291851 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.293364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.293859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:40.294271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:40.294909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.295498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:40.295498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:40.295498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.295498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:40.295498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.295498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:40.957373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:40.959830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:40.959830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:40.959830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.959830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.959830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.960321 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.961653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.962108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:40.962448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.962448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:40.963663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:40.964182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.964693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:40.964693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.964693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:40.966153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:40.966693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:40.966693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:40.966857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:40.967385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.967558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.968615 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:40.968910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.969266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.970929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:40.971952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:40.972791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.973725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:40.975567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:40.976404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:40.976404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:40.976404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:40.978272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.978597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.979267 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:40.980023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.980581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.981018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:40.981394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:40.981799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.981799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:40.981799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.981799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.982324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:40.982861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:40.983368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.983368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:40.983368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.983368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:40.986682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:40.987084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:40.987201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:40.987368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:40.987938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.988214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:40.988375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.988997 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.989858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.990237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:40.990571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.990571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:40.990857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:40.991313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:40.991863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:40.991863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:40.991863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:45.253979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:45.254617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:45.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:45.256281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:45.301491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.301742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.302458 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.303940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.304426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:45.305292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:45.305848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.306410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:45.306410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:45.306410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.306410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:45.306410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.306410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:45.958021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:45.959370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:45.959563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:45.960553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.960828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.960980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.961591 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.962629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.962932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:45.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.963276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:45.963821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:45.964300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.964736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:45.964736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.964736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:45.966630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:45.967185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:45.967324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:45.967457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:45.967789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.967915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.968348 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:45.970547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.971018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.971425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:45.971792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:45.972178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.972178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:45.972178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.972178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:45.972862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.973478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.973956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:45.976096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:45.977249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:45.977249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:45.977249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:45.977249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.977436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.977844 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:45.979430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.979996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.980454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:45.980840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:45.981274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.981274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:45.981274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.981274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.981741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:45.982274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:45.982835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.982835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:45.982835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.982835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:45.989012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:45.989515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:45.989653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:45.989863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:45.990193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.990455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:45.990552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.991097 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.992513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.993177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:45.993775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:45.994577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:45.995404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:45.995404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:45.995404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:45.995404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:45.995404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:45.995404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:50.254665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:50.255603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:50.255603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:50.256453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:50.301533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.301807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.302712 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.305139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.305563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:50.305978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.305978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:50.306183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:50.306677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.307214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:50.307214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.307214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:50.959393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:50.959656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:50.959816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:50.960409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.960504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.960658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.965742 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.968675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.969094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:50.969463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.969463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:50.969463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:50.970433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:50.970988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.971529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:50.971529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.971529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:50.971529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:50.971529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:50.971529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:50.975211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.975450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.976247 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:50.977886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:50.978384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.979439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:50.980050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:50.980660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.981077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:50.981557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.981928 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:50.983137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:50.983634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:50.984360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:50.985007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:50.985614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.987678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:50.989483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:50.989842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:50.990024 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:50.990184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:50.990634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.990950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:50.991305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.991707 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.992842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.993127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:50.993393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.993393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:50.994039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:50.994485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:50.994990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:50.994990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:50.994990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:55.255591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:55.256507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:55.256507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:55.257409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:55.291596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.292020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.293050 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.294526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.295142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:55.295705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.295705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:55.296973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:55.297738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.300013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:55.300013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.300013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:55.959962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:55.960433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:55.960628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:55.961438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.961743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.962022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.963029 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.963802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.964275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:55.964806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.964806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:55.964907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:55.965463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.966413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:55.966413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.966413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:55.971237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:55.971504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:55.971620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:55.971805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:40:55.972217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.972437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.973030 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:55.974099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.974625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.975468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:55.976570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:55.977154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.977667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.978312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:55.980885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:55.981534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:55.981657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:55.981822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:55.982628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.983520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.984367 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:40:55.985213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.985718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.986160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:55.986967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:55.987702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:55.988578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.989550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:40:55.989941 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:40:55.990202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:40:55.990202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:40:55.990434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:40:55.991132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.991672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:40:55.991955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.992581 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.993394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.993853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:40:55.994282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.994282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:40:55.994356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:40:55.994871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:40:55.995608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:40:55.995608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:40:55.995608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:00.256260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:00.258611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:00.258611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:00.258611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:00.287919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.288262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:00.290426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:00.290810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.291173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:00.291467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.291927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:00.960681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:00.962244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:00.962244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:00.962930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.963185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.963360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.964012 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.967828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.968281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:00.968622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:00.969138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.969702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:00.969702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:00.969702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.969702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:00.969702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.969702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:00.971575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:00.971836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:00.972030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:00.972213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:00.972692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.972943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.973844 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:00.974627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:00.975335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.975798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.976203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:00.976574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.977084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.977646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:00.981682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:00.981961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:00.982065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:00.982196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.985277 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:00.986517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:00.988188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:00.988961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:00.989384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:00.989708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.990426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:00.991881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:00.991881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:00.991881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:00.991881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:00.992811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.993034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:00.993255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.994021 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.994841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.996049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:00.996615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:00.997074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:00.997641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:00.997641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:00.997641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:00.997641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:00.997641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:00.997641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:05.257489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:05.258262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:05.258262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:05.258811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:05.296437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.296851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:05.299809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:05.300565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.301257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:05.301946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.302884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:05.963954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:05.964511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:05.964626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:05.965537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.965723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.965893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.966470 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.967412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.968059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:05.968688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:05.969646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.971355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:05.971355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:05.971355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.971355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:05.971355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.971355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:05.972114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:05.972719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:05.972993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:05.973201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:05.973678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.973897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.974497 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:05.975711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.976390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.976889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:05.977463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:05.978602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.979252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.979964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:05.982079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:05.982308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:05.982390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:05.982490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:05.983047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.983248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.983686 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:05.984352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.984633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.985059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:05.985299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:05.985650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.985650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:05.985650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.985650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.985897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:05.986401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:05.986728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.986728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:05.986728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.986728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:05.991520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:05.991916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:05.992083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:05.992343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:05.993452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.993638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:05.993740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.995579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.996044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:05.996377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.996377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:05.996988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:05.997432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:05.997964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:05.997964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:05.997964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:10.258099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:10.258904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:10.258904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:10.259460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:10.299130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.299285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.299915 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.300632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.301285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:10.301626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:10.302045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.303031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:10.303031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:10.303031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.303031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:10.303031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.303031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:10.964813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:10.965646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:10.965855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:10.967016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.967353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.967606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.968986 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.970074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.970423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:10.970738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.970738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:10.971178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:10.971729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.972198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:10.972198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.972198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:10.972618 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:10.972926 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:10.973953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:10.973953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:10.973953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.974168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.974832 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:10.975631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.976248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.976612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:10.977075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:10.977554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.977554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:10.977554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.977554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:10.977710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.978303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.978942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:10.982535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:10.982849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:10.982955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:10.983216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:10.983567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.984393 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:10.985198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.985566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.986135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:10.986459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:10.986825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.986825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:10.986825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.986825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.989792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:10.990404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:10.991214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.991214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:10.991214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.991214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:10.992087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:10.995058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:10.995156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:10.995324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:10.995921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.997913 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:10.998507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:10.998885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:10.999226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:10.999226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:11.000577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:11.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:11.002203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:11.002203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:11.002203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:15.258889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:15.259442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:15.259644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:15.260970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:15.295341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.295535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.296101 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.296891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.297357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:15.297831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:15.298296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.298922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:15.298922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:15.298922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.298922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:15.298922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.298922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:15.965189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:15.966913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:15.967120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:15.968381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.969031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.970183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.970734 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.971775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.972190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:15.972599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.972599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:15.972912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:15.973484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:15.974165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.974421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.974611 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:15.975366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.975855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.976221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:15.976746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:15.977280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.977674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.978254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:15.984525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:15.984774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:15.984905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:15.985026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:15.985514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.985996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.986858 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:15.988097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.988559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.989886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:15.990229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:15.990561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.990561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:15.990561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.990561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.992378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:15.992924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:15.993433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.993433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:15.993433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.993433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:15.995609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:15.995609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:15.995795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:15.995994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:15.996689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.997088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:15.997296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.997931 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:15.998831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:15.999375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:16.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:16.001036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:16.001700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:16.001700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:16.001700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:16.001700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:16.001700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:16.001700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:20.259413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:20.260609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:20.260609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:20.261944 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:20.263806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=434 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:41:20.310635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.310925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.311604 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.312529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:20.314056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:20.314803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.316118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:20.316118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:20.316118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.316118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:20.316118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.316118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:20.965785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:20.966345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:20.966618 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:20.967689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.968181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.968378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.968949 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.970247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.970864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:20.971394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:20.972077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.973122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:20.973122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:20.973122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.973122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:20.973122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.973122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:20.974741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:20.975177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:20.975342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:20.975513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:20.976009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.976202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.976991 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:20.978185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.978598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.979318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:20.979690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:20.980062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.980062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:20.980062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.980062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:20.980740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.981163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.981663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:20.984860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:20.985123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:20.985239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:20.985369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:20.987127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.987374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.987971 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:20.988729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.989141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.989977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:20.990277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:20.990632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.990632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:20.990632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.990632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.991795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:20.992714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:20.993638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.993638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:20.993638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.993638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:20.995552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:20.995914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:20.996099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:20.996336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:20.996783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.997034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:20.997187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.997745 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:20.998723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:20.999330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:21.000114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:21.000658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:21.001210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:21.001210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:21.001210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:21.001210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:21.001210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:21.001210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:25.260083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:25.260858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:25.260975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:25.262020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:25.309694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.309888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.310414 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.311386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.311847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:25.312286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.312286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:25.312424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:25.312980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.313710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:25.313710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.313710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:25.966346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:25.966983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:25.967183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:25.968331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.968651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.968911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.969812 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.971447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.971914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:25.972373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.972373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:25.973480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:25.973962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.974496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:25.974496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.974496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:25.975140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:25.975503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:25.975616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:25.975745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:25.976078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.976220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.976618 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:25.977378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.977858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.978186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:25.978626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:25.979122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.979122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:25.979122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.979122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:25.979240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.979750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.980295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:25.985870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:25.986151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:25.986256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:25.986385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:25.986920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.987226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.987797 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:25.988560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.989273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.989603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:25.990128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:25.990648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:25.991351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.992082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:25.996286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:25.996559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:25.996685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:25.996887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:25.997357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.997598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:25.997756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.998321 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:25.999326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:26.000252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:26.000793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:26.001423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:26.002167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:26.002167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:26.002167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:26.002167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:26.002167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:26.002167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:30.260749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:30.261695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:30.261899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:30.263794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:30.303488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.303789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.304557 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.306181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.306512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:30.307042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.307042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:30.307888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:30.309310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.309969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:30.309969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.309969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:30.967033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:30.968068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:30.968434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:30.969660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.969984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.970256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:30.974836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:30.975646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.976051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:30.976051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:30.976051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.976051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:30.976051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.976051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:30.976536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:30.976996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.977575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:30.977575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.977575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:30.977992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:30.978961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:30.978961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:30.978961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:30.979326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.979541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.982210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:30.982602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.982944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.983178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:30.983434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:30.983707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.984063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:30.986297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:30.986552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:30.986727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:30.986986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:30.987444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.987695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.989892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:30.990424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.990739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.991048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:30.991977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:30.992663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:30.993371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.993371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:30.993371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.993371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:30.996647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:30.997113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:30.997232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:30.997461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:30.997836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.998070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:30.998220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:30.999382 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:31.001114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:31.001496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:31.001799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:31.001799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:31.003123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:31.003472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:31.003841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:31.003841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:31.003841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:35.261567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:35.262214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:35.262214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:35.263088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:35.295169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.295313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.295846 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.296813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.297121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:35.297359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.297359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:35.298002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:35.298341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.298681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:35.298681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.298681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:35.967969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:35.968610 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:35.968932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:35.969849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.969995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.970103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.970668 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.972583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.973228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:35.973574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.973574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:35.974942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:35.975499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.976165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:35.976165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.976165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:35.978566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:35.979014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:35.979190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:35.979381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:35.979775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.979981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.980534 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:35.981619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.982303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.982886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:35.983336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:35.984120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.984120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:35.984120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.984120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:35.984624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.985379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.986037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:35.986775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:35.987134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:35.987282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:35.987464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:35.987912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.988629 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:35.989327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.989873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.990165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:35.990449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:35.990828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.990828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:35.990828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.990828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.990907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:35.991222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:35.991547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.991547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:35.991547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.991547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:35.997003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:35.997267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:35.997381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:35.997498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:35.997732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.997919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:35.998023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.998378 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:35.999215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:35.999713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:36.000189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:36.000189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:36.000291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:36.000716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:36.001220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:36.001220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:36.001220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:40.262265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:40.263801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:40.263801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:40.264118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:40.298725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.298994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.299668 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.300409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.300795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:40.301083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.301083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:40.301166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:40.301594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.302088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:40.302088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.302088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:40.968489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:40.968924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:40.969126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:40.970279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.970710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.971068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.971944 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.973058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.973599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:40.974124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.974124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:40.974545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:40.975131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.975617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:40.975617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.975617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:40.978988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:40.979280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:40.979393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:40.979524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:40.980057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.980280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.980869 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:40.981465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:40.982290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.982936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.983503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:40.984037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.984703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.985484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:40.987218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:40.987496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:40.987571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:40.987673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:40.988154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.988407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.988990 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:40.989581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.990066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.990324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:40.990677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:40.991084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:40.991561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:40.992176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:40.997419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:40.997814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:40.997886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:40.998072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:40.998777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.999279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:40.999569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:41.000406 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:41.001593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:41.002021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:41.002393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:41.002393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:41.003406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:41.003892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:41.004628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:41.004628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:41.004628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:45.262881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:45.265634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:45.265634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:45.265634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:45.299903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.300811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.301164 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.302819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.303311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:45.303815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.303815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:45.305202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:45.305613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.306070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:45.306070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.306070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:45.969166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:45.970699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:45.970699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:45.971417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.972055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.972294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:45.975365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:45.976482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.977032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:45.977032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:45.977032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.977032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:45.977032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.977032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:45.977723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:45.978319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.978922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:45.978922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.978922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:45.979484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:45.980307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:45.980307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:45.980307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:45.980693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.980935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.983819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:45.984740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.985713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.986347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:45.987320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:45.988073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:45.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:45.989097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:45.989097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.989097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.989713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:45.990614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.991241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.992024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:45.994038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:45.994375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:45.994744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:45.995210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:45.998014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:45.998314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:45.998433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:45.998603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:45.999175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.999450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:45.999648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:46.000245 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:46.001176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:46.001546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:46.001890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:46.001890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:46.003476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:46.003992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:46.004664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:46.004664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:46.004664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:50.264463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:50.265351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:50.265351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:50.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:50.299897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.300117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.301254 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.302309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:50.303036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.303036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:50.303795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:50.304226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.304687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:50.304687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.304687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:50.970067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:50.970542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:50.970663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:50.971567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.971793 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.971938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.972702 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.973748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.974218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:50.974590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.974590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:50.974874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:50.975349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.975899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:50.975899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.975899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:50.979987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:50.980265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:50.980386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:50.980514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:50.982826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.983329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:50.983848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:50.984217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.984584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.985034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.985524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:50.988055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:50.988315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:50.988427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:50.988635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:50.989157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.989385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.989879 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:50.990619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.991065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.991471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:50.991829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:50.992201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.992201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:50.992201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.992201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:50.992938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:50.993531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:50.994166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:50.994166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:50.994166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:50.994166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:50.998513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:50.998765 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:50.998893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:50.999045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:50.999498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.999711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:50.999839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:51.000318 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:51.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:51.001628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:51.001975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:51.001975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:51.003787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:51.004222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:51.004663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:51.004663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:51.004663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:55.265161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:55.266150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:55.266150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:55.266614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:55.306260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:55.306535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.307308 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.308526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.309022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:55.309443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.309443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:55.310422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:55.311559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.312145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:55.312145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.312145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:55.970798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:55.972085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:55.972267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:55.974166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.975276 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.978645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.979127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:55.979575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.979575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:55.979664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:55.980226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.980859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:55.980859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.980859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:55.982233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:55.982718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:55.982914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:55.983019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:41:55.983490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:55.983775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.984571 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:55.985371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:55.986368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.986869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.987214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:55.987531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.987987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:55.988547 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:55.989674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:55.989674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:55.989674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:55.989674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:55.989674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.990132 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:41:55.990636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:55.991614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:55.992646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:55.993599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:41:55.994647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:55.995118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:41:55.998904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:41:55.999236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:41:55.999385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:41:55.999603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:41:56.000164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:56.000572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:41:56.000856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:56.001627 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:41:56.002684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:56.003063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:41:56.003401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:56.003401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:41:56.003901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:41:56.004466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:41:56.005038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:41:56.005038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:41:56.005038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:00.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:00.266977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:00.267126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:00.268028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:00.308959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:00.309146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.309537 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.310898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.311445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:00.311938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:00.312419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.313058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:00.313058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:00.313058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.313058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:00.313058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.313058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:00.972047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:00.974074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:00.974074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:00.974365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:00.974561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:00.974712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.975404 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.976548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.977348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:00.977671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:00.978053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.978594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:00.978594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:00.978594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.978594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:00.978594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.978594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:00.983349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:00.983744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:00.983859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:00.983960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:00.984268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:00.984434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.984958 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:00.985568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.986080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.986323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:00.986673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:00.987052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.987052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:00.987052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.987052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:00.987190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.987617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.988120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:00.988923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:00.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:00.989482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:00.989579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:00.991909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.992466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:00.992949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.993375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:00.993737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:00.993737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.993737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:00.993737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.993737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:00.993893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:00.994376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:00.995029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:00.995029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:00.995029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:00.995029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:01.001871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:01.001871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:01.001871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:01.001871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:01.001871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:01.001871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:01.002286 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:01.003252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:01.003561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:01.003882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:01.003882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:01.004930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:01.005349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:01.005910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:01.005910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:01.005910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:05.268579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:05.268579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:05.268579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:05.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:05.310570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:05.311219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.311636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:05.312124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.312966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:05.973665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:05.973976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:05.974044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:05.974747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:05.975100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:05.975331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.975958 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:05.977492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:05.977903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:05.978268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.978268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:05.978974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:05.979444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:05.980030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:05.980030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:05.980030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:06.012124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:06.012331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:06.012387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:06.013480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:06.013689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:06.014089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:06.015159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:06.016589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.019191 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:06.020616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.021438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.023253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:06.023821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:06.024982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.024982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:06.024982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.024982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:06.025533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.025958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.026321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:06.026450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:06.027355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:06.027355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:06.027355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:06.027355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.028308 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:06.029040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.029637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.030727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:06.031280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:06.031828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.031828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:06.031828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.031828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:06.033911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:06.034495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.034791 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.035693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.036117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:06.036425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.036425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:06.036579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:06.037462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:06.037952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:06.037952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:06.037952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:10.267714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:10.268397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:10.268397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:10.269201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:10.306261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:10.306509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.307115 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:10.308268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:10.308693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:10.309139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.309139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:10.310039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:10.310567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:10.311519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:10.311519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.311519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:10.974404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:10.975019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:10.975112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:10.975704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:10.976134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:10.977122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.977826 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:10.978467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:10.978983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:10.979421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:10.979898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:10.980459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:10.980459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:10.980459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:10.980459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:10.980459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:10.980459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:11.013053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:11.013502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:11.013617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:11.013779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:11.014233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:11.014506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.015221 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:11.016239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.016817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.017745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:11.018122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:11.018426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.018426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:11.018426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.018426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:11.019241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.019662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.020025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:11.026219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:11.032707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:11.045606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:11.045701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:11.045862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:11.046194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:11.046435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.048861 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:11.049815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.050099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.051299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:11.051566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:11.051873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.051873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:11.051873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.051873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.052422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:11.052746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:11.053259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:11.053499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:11.054082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.054365 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.055327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.055927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:11.056429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:11.056844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:11.057551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:11.057551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:11.057551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:11.057551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:11.057551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:11.057551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:15.268262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:15.268935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:15.269061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:15.270096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:15.309452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:15.309706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.310417 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:15.311553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:15.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:15.312636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:15.313204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:15.313634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:15.313634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:15.313634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:15.313634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:15.313634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.313634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:15.975259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:15.975899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:15.976151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:15.977189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:15.977480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:15.977696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.978580 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:15.980077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:15.980682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:15.981277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.981277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:15.981868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:15.983868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:15.984384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:15.984384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:15.984384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:16.013595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:16.014125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:16.014276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:16.014473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:16.015682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:16.015682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.016302 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:16.017845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.018254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.019106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:16.020164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:16.020676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.021032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.022513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:16.045792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:16.046230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:16.046230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:16.046388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:16.047203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:16.047726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.048602 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:16.050007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:16.051820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:16.052945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:16.053480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:16.054331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.055733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:16.057158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:16.057158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:16.057158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:16.057158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:16.057158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:16.057158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.058316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:16.059536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:16.060349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:16.060837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:20.269217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:20.270114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:20.270114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:20.270640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:20.273141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:42:20.304088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:20.304317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.305030 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:20.305785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:20.306430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:20.307098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:20.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:20.309169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:20.309169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:20.309169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:20.309169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:20.309169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.309169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:20.975902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:20.976157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:20.976219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:20.976952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:20.977071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:20.977162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.978512 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:20.978975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:20.979493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:20.980126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:20.980788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:20.981671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:20.981671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:20.981671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:20.981671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:20.981671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:20.981671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:21.014131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:21.014497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:21.014706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:21.014937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:21.015280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:21.015451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.016422 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:21.017627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.018557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.019070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:21.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:21.020408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.020408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:21.020408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.020408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:21.021128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.021878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.022885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:21.046355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:21.046970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:21.047138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:21.047360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:21.048291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:21.048573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.049223 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:21.050135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.050879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.051816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:21.052469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:21.053281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.053281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:21.053281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.053281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:21.054737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:21.055661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:21.056162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:21.056162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:21.056162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.057476 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.057910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.058179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:21.058429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.058429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:21.058611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:21.058936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:21.059299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:21.059299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:21.059299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:25.269873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:25.270475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:25.270709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:25.272095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:25.311593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:25.312174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.313307 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:25.315333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:25.315815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:25.316155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.316155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:25.316253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:25.316722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:25.317277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:25.317277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.317277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:25.976629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:25.977150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:25.977283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:25.978092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:25.978441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:25.978711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.981039 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:25.982119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:25.982740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:25.983332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:25.983988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:25.984869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:25.984869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:25.984869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:25.984869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:25.984869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:25.984869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:26.014702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:26.015212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:26.015321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:26.015438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:26.015937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:26.016192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.017076 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:26.017792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.018238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.018642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:26.019069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:26.019524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.019984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.020475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:26.047058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:26.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:26.047669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:26.047837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:26.049937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:26.050371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.051321 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:26.052015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:26.052809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:26.053498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:26.054154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:26.054869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.056146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:26.057370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:26.057370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:26.057370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:26.057370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:26.057370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:26.057370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.057556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.057979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:26.058524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:26.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:26.059804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:30.271460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:30.272453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:30.272453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:30.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:30.308025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:30.308407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.309039 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:30.309881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:30.310449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:30.311007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:30.311706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:30.312139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:30.312139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:30.312139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:30.312139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:30.312139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.312139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:30.977279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:30.977665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:30.977744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:30.978547 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:30.978802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:30.978994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.979567 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:30.980409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:30.980941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:30.981447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:30.982042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:30.982969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:30.982969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:30.982969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:30.982969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:30.982969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:30.982969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:31.015378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:31.016158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:31.016380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:31.016380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:31.016921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:31.017171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.017855 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:31.018679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.019236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.019772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:31.020235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:31.020599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.020599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:31.020599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.020599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:31.020738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.021215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.021713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:31.047714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:31.048372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:31.048372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:31.048683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:31.049143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:31.049400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.050096 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:31.050710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:31.051536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:31.052300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:31.053080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:31.053752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.055634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:31.056648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:31.056648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:31.056648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:31.056648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:31.057394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:31.057800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:31.058016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.060804 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.061879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:31.062260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:31.062697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:31.063262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:31.063262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:31.063262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:31.063262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:31.063262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:31.063262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:35.272113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:35.272747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:35.272858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:35.274047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:35.315798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:35.315798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:35.316924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:35.317338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:35.317694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:35.318194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:35.318575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:35.318575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:35.318575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:35.318575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:35.318575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.318575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:35.977895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:35.978190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:35.978278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:35.979096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:35.979780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:35.979780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.980646 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:35.981474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:35.981791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:35.982029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.982029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:35.982276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:35.982612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:35.983004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:35.983004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:35.983004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:36.016263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:36.016918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:36.017141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:36.017387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:36.018313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:36.018616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.019639 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:36.020785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.021316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.021948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:36.022447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:36.022943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.022943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:36.022943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.022943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:36.023322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.024028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.024921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:36.050511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:36.051059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:36.051254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:36.051510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:36.052331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:36.052804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.053696 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:36.055096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.055621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.056775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:36.057148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:36.057582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.057582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:36.057582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.057582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:36.057582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.059022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:36.059510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:36.060112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:36.060412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:36.060666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:36.060939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.061490 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.062438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.062804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:36.063156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.063156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:36.063753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:36.064346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:36.064950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:36.064950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:36.064950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:40.272820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:40.273545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:40.273545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:40.274997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:40.314417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:40.314973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.315728 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:40.316519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:40.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:40.317408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:40.317920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:40.318503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:40.318503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:40.318503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:40.318503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:40.318503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.318503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:40.978584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:40.979093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:40.979233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:40.980218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:40.980467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:40.980637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.981344 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:40.982169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:40.982828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:40.983544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:40.984485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:40.985396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:40.985396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:40.985396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:40.985396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:40.985396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:40.985396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:41.017340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:41.018396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:41.018692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:41.018954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:41.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:41.020104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.020883 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:41.021991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.022471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.023006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:41.023459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:41.023963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.023963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:41.023963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.023963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:41.024268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.024746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.025388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:41.051079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:41.051546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:41.051667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:41.051830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:41.052431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:41.052702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.053353 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:41.054131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.054616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.055136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:41.055496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:41.055933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.055933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:41.055933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.055933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.056285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:41.056847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:41.057417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.057417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:41.057417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.057417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:41.059445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:41.059806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:41.059936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:41.060122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:41.060708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:41.061087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:41.061281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.061995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.062985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.063383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:41.063795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.063795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:41.064090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:41.064557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:41.065111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:41.065111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:41.065111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:45.273417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:45.274075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:45.274075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:45.274631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:45.309576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:45.309863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.310438 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:45.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:45.311675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:45.312060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.312060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:45.312212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:45.312713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:45.313496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:45.313496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.313496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:45.979219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:45.979396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:45.979471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:45.980025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:45.980165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:45.980280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.980722 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:45.981431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:45.981838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:45.982170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:45.982631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:45.983118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:45.983118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:45.983118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:45.983118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:45.983118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:45.983118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:46.020077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:46.020476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:46.020476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:46.020586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:46.021067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:46.021288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.022414 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:46.023165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.023836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.024142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:46.024526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:46.024967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.024967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:46.024967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.024967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:46.025087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.025576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.026232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:46.051737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:46.051961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:46.052021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:46.052145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:46.052594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:46.053046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.055235 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:46.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.057011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.057914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:46.058287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:46.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:46.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.059924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:46.060618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:46.061420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.061688 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.062563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.063492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:46.064382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:46.065306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:46.067872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:46.067872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:46.067872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:46.067872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:46.067872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:46.067872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:50.274084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:50.274915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:50.274915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:50.275617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:50.311519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:50.311711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.312518 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:50.313891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:50.314477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:50.315179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.315179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:50.315495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:50.316207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:50.317113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:50.317113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.317113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:50.979956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:50.980426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:50.980622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:50.981512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:50.981831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:50.982133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.983552 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:50.984497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:50.984886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:50.985232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.985232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:50.985593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:50.986033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:50.986544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:50.986544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:50.986544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:51.020721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:51.021139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:51.021322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:51.021507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:51.021978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:51.022267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.023148 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:51.024292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.024990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.025698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:51.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:51.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:51.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:51.027291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.027613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.028041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:51.052244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:51.052658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:51.052738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:51.052917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:51.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:51.053601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.054392 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:51.055218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.055646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.056132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:51.056535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:51.057050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.057050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:51.057050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.057050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.057263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:51.058024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:51.058918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.058918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:51.058918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.058918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:51.060246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:51.060523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:51.060710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:51.060968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:51.061243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:51.061486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:51.061624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.062148 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.063013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.063541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:51.064245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:51.064888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:51.065666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:51.065666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:51.065666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:51.065666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:51.065666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:51.065666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:55.274796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:55.275419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:55.275580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:55.276975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:55.314544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:55.314735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.315147 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:55.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:55.316618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:55.317247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:55.317893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:55.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:55.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:55.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:55.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:55.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:55.980814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:55.981666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:55.982026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:55.983224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.985490 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:55.986295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:55.986899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:55.987459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.987459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:55.988181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:55.988671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:55.989281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:55.989281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:55.989281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:56.021345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:56.023872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:56.023872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:56.023872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:42:56.023872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:56.023872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.024492 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:56.025407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.025697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.026334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:56.026571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:56.026842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.026842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:56.026842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.026842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:56.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.028713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.029058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:56.052892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:56.053323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:56.053449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:56.053590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:56.054139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:56.054412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.054981 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:42:56.055912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.056337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.057405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:56.057874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:56.058240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.058240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:56.058240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.058240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.059031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:56.059525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:56.060069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.060069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:56.060069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.060069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:42:56.063791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:42:56.063791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:42:56.063791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:42:56.063791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:42:56.063791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:56.063791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.064038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.064452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.064876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:42:56.066011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:42:56.066321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:42:56.066681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:42:56.066681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:42:56.066681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:00.275470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:00.276208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:00.276208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:00.276652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:00.312841 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:00.313312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.314259 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:00.315039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:00.315896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:00.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:00.317696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:00.318603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:00.318603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:00.318603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:00.318603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:00.318603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.318603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:00.981797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:00.982982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:00.983388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:00.984477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:00.984940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:00.985132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.987100 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:00.988063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:00.988795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:00.989502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:00.990238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:00.991080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:00.991080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:00.991080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:00.991080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:00.991080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:00.991080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:01.021960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:01.022403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:01.022494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:01.022620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:01.023649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:01.024002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.024788 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:01.025627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:01.026548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.027376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.028079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:01.028623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.030491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.031370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:01.053450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:01.054903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:01.055149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:01.055576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:01.056082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:01.056436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.057265 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:01.058237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.059129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:01.059457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:01.059819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.059819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:01.059819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.059819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.060499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:01.061554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.061554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:01.061554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.061554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:01.061554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:01.061936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:01.062051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:01.062290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:01.063342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:01.063776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:01.064033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.064695 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.065686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.066117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:01.066487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.066487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:01.066713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:01.067215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:01.067748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:01.067748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:01.067748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:05.276165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:05.276857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:05.277073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:05.278319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:05.313734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:05.313734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.313993 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:05.314670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:05.315200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:05.315722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:05.316374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:05.317116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:05.317116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:05.317116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:05.317116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:05.317116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.317116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:05.982609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:05.983502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:05.983502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:05.986878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:05.986878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:05.986878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.987520 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:05.989392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:05.989955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:05.990574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.990574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:05.992090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:05.993336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:05.994333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:06.023790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:06.023790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:06.023790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:06.023790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:06.025303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:06.025590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.026202 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:06.026953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.027361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.027576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:06.027861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:06.028156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.028156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:06.028156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.028156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:06.029242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.029649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.030057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:06.054053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:06.054606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:06.054736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:06.054898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:06.055561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:06.055900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.056492 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:06.057331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.057987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.058319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:06.058819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:06.059269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:06.059630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.060231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:06.062053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:06.062342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:06.062443 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:06.062639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:06.063103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.064865 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.065136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.065398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:06.065650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.065650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:06.066051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:06.066415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:06.066922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:06.066922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:06.066922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:10.276909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:10.277683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:10.277683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:10.278138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:10.315542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:10.315861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.316574 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:10.317632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:10.317962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:10.318219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.318219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:10.318743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:10.319513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:10.320571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:10.320571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.320571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:10.983190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:10.983729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:10.983919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:10.985015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:10.985298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:10.985537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.986271 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:10.987416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:10.987854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:10.988263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.988263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:10.988564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:10.989146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:10.989719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:10.989719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:10.989719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:11.023170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:11.023949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:11.024299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:11.024476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:11.025136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:11.025411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.026140 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:11.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:11.030090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.030840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.031482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:11.032046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.032793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.033924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:11.054631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:11.055154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:11.055347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:11.055589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:11.056268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:11.056612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.057298 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:11.058602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.059077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.060035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:11.060397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:11.060835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.060835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:11.060835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.060835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.061713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:11.062322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:11.062951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.062951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:11.062951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.062951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:11.062951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:11.062951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:11.063182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:11.063278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:11.063946 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:11.064404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:11.064697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.065619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.067062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.067717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:11.068168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:11.068651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:11.069273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:11.069273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:11.069273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:11.069273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:11.069273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:11.069273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:15.277372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:15.278306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:15.278306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:15.279261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:15.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:15.323950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.324631 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:15.325555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:15.326279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:15.326984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:15.327816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:15.328926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:15.328926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:15.328926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:15.328926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:15.328926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.328926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:15.983933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:15.984206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:15.984430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:15.985692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:15.986015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:15.986189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.987037 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:15.988193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:15.988654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:15.989022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.989022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:15.989338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:15.990323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:15.990967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:15.990967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:15.990967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:16.023811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:16.024352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:16.024563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:16.024739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:16.025444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:16.025721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.026340 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:16.027378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.028011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.028701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:16.029251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:16.029921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.029921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:16.029921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.029921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:16.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.031166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.031700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:16.055197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:16.055508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:16.055626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:16.055771 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:16.056570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:16.057093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.058175 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:16.059011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.059579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.059947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:16.060382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:16.061020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:16.061558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.062266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:16.062806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:16.062997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:16.063106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:16.063241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:16.063769 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:16.064144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:16.064386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.065100 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.065933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.066458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:16.066937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:16.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:16.068100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:16.068100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:16.068100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:16.068100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:16.068100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:16.068100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:20.278086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:20.278616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:20.278616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:20.279371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:20.281425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:43:20.316551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:20.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.317948 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:20.319253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:20.319649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:20.320211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.320211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:20.320582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:20.321089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:20.321672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:20.321672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.321672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:20.984301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:20.984543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:20.984640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:20.985466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:20.985852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:20.986054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.986859 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:20.988542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:20.989412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:20.990309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.990309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:20.990702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:20.991746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:20.993024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:20.993024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:20.993024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:21.024459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:21.024732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:21.024866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:21.024987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:21.028159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:21.028360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.028922 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:21.029546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:21.030330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.031045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.031625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:21.032181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.032913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:21.055950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:21.056287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:21.056464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:21.056659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:21.058715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:21.059086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.059721 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:21.060622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.061254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.061966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:21.062492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:21.063086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.063086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:21.063086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.063086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.063706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:21.064517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:21.065103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:21.065354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:21.068348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:21.068550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.070198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.070748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:21.071304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.071304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:21.071638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:21.072209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:21.072752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:21.072752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:21.072752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:25.278656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:25.279409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:25.279409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:25.280284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:25.319165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:25.319332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.319938 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:25.320881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:25.321284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:25.321657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.321657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:25.322017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:25.322466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:25.322991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:25.322991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.322991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:25.984918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:25.985362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:25.985528 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:25.986243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:25.986387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:25.986502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.987013 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:25.988387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:25.988799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:25.989265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.989265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:25.990153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:25.990623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:25.991144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:25.991144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:25.991144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:26.025227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:26.025719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:26.026026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:26.026247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:26.026621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:26.026790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.027284 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:26.028392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.028828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.029695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:26.030094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:26.030470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.030470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:26.030470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.030470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:26.031526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.033223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:26.059812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:26.059812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:26.059812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:26.059812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:26.059812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:26.059812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.060131 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:26.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.060852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.062047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:26.062377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:26.062743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.062743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:26.062743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.062743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.064205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:26.064737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:26.065571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:26.066143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:26.066205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:26.066307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.066890 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.067956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.068358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:26.068726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.068726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:26.069123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:26.069638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:26.070386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:26.070386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:26.070386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:30.279189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:30.279965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:30.279965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:30.280538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:30.313456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:30.313680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.314225 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:30.315009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:30.315461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:30.315929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:30.316352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:30.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:30.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:30.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:30.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:30.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.316901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:30.986428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:30.986428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:30.986428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:30.987128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:30.987318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:30.987551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.988416 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:30.989256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:30.989897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:30.990618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:30.991302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:30.992081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:30.992081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:30.992081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:30.992081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:30.992081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:30.992081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:31.025963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:31.026326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:31.026560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:31.026797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:31.027627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:31.028347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.028583 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:31.030253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.030745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:31.032167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:31.032443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.032443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:31.032443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.032443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:31.035889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.036479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.037039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:31.057105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:31.057450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:31.057572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:31.057701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:31.058100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.059545 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:31.059791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.060130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.061293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:31.061563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:31.061850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:31.062128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.062525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:31.064972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:31.065250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:31.065360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:31.065541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:31.067104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:31.067352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:31.067494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.067984 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.068842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.069124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:31.069384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.069384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:31.069871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:31.070171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:31.070502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:31.070502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:31.070502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:35.279809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:35.280482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:35.280482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:35.281969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:35.316655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:35.316971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.317526 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:35.318308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:35.318643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:35.318999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.318999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:35.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:35.321086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:35.322548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:35.322548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.322548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:35.987812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:35.987812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:35.987812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:35.988104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:35.988328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:35.988539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.990734 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:35.993331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:35.993762 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:35.994132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.994132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:35.995549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:35.996070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:35.996625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:35.996625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:35.996625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:36.026587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:36.027077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:36.027252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:36.027445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:36.028333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:36.028593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.029469 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:36.030146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.030647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.030921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:36.031285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:36.031646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.031646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:36.031646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.031646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:36.031790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.032218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.032693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:36.057740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:36.058016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:36.058088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:36.058226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:36.058734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:36.058961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.060329 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:36.062263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.062732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.063741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:36.064128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:36.064499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.064499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:36.064499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.064499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.065559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:36.066218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:36.066765 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:36.067415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:36.067707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:36.067959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.068638 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.069748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.070147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:36.070483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.070483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:36.070752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:36.071210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:36.071704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:36.071704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:36.071704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:40.280473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:40.281256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:40.281256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:40.281588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:40.318663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:40.319882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.320904 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:40.323984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:40.324743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:40.325321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:40.326013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:40.328866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:40.328866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:40.328866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:40.328866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:40.328866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.328866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:40.987088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:40.987841 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:40.988187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:40.989138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:40.989331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:40.989709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.990497 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:40.992003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:40.992514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:40.992996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.992996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:40.993594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:40.994243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:40.994947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:40.994947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:40.994947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:41.027178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:41.027651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:41.027752 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:41.027875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:41.028197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:41.028709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.028981 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:41.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.030109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.030487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:41.030733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:41.031007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.031007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:41.031007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.031007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:41.031324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.031636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.032007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:41.058411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:41.058921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:41.059051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:41.059941 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:41.059941 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:41.060102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.063144 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:41.064316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:41.068921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:41.069601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:41.070324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:41.071193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.072464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:41.073713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:41.073713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:41.073713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:41.073713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:41.073713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:41.073713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.074290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.074868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:41.075145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:41.075404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:41.075784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:45.280992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:45.282964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:45.282964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:45.283195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:45.321503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:45.321716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.322268 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:45.323187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:45.323877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:45.324724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:45.325708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:45.326467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:45.326467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:45.326467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:45.326467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:45.326467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.326467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:45.988176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:45.991809 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.992347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:45.992841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:45.993233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:45.993233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:45.993233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.993233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:45.993850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:45.994244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:45.994652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:45.994652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:45.994652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:46.027891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:46.028421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:46.028558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:46.028718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:46.029080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:46.029244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.029694 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:46.030542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.030864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.031466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:46.031724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:46.032047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.032047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:46.032047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.032047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:46.032681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.033071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.033390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:46.059032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:46.059459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:46.059548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:46.059646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:46.060161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:46.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.061268 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:46.061833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.062131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.062781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:46.063015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:46.063277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.063277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:46.063277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.063277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.065427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:46.065863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:46.066253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.066253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:46.066253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.066253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:46.067626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:46.068007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:46.068166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:46.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:46.070031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:46.070221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:46.070339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.070804 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.072149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.072523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:46.072796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.072796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:46.073856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:46.074173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:46.074501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:46.074501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:46.074501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:50.282144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:50.282717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:50.282717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:50.283561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:50.322813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:50.322813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.323105 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:50.323918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:50.324223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:50.324486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.324486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:50.324719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:50.325079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:50.325492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:50.325492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.325492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:50.988925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:50.989442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:50.989577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:50.990593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:50.990938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:50.991160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.991936 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:50.993128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:50.993627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:50.994079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.994079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:50.994179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:50.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:50.995249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:50.995249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:50.995249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:51.028477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:51.028970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:51.029079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:51.029266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:51.030014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:51.030375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.031172 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:51.031927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.032434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.033218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:51.033650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:51.034106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.034106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:51.034106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.034106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:51.035335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.035833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.036391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:51.059635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:51.063937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:51.063937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:51.063937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:51.064433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:51.064713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.065334 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:51.066602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.067073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.067419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:51.067783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:51.068161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.068161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:51.068161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.068161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:51.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:51.070633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.070633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:51.070633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.070633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:51.072002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:51.072393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:51.072498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:51.072682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:51.073160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:51.073349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:51.073465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.073950 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.074812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.075427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:51.076062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:51.076590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:51.077203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:51.077203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:51.077203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:51.077203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:51.077203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:51.077203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:55.282901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:55.283503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:55.283503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:55.284252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:55.324541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:55.325602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.326496 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:55.327340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:55.327708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:55.328069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:55.328453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:55.328948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:55.328948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:55.328948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:55.328948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:55.328948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.328948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:55.989530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:55.990087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:55.990280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:55.991269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:55.991522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:55.991800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.992438 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:55.993316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:55.993666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:55.994047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.994047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:55.994213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:55.994649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:55.995150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:55.995150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:55.995150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:56.029075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:56.029525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:56.029623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:56.029691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:43:56.030290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:56.030516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.031208 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:56.031990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.032603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.032918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:56.033319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:56.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.034134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.034738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:56.063303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:56.063927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:56.064182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:56.064365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:56.065266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:56.065676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.068102 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:43:56.068874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.069290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.069641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:56.070056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:56.070482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.070482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:56.070482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.070482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.070745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:56.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:56.071949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.071949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:56.071949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.071949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:43:56.072553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:43:56.072786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:43:56.072884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:43:56.073009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:43:56.073620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:56.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:43:56.074148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.075401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:43:56.076258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:43:56.076694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:43:56.077262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:43:56.077262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:43:56.077262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:43:56.077262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:43:56.077262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:43:56.077262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:00.283425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:00.284334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:00.284334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:00.286327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:00.332064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:00.332328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:00.335034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:00.335552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:00.335889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:00.335889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:00.335889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:00.335889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:00.335889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.335889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:00.336986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:00.337424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:00.337824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:00.337824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.337824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:00.990118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:00.991439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:00.991439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:00.991729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:00.991929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:00.992039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:00.994811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:00.995463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:00.995992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:00.996574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:00.997382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:01.029598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:01.030097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:01.030273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:01.030441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:01.031211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:01.031503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.034671 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:01.035502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.036196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.038382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:01.038779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:01.039116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.039511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.040007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:01.063875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:01.064433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:01.064689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:01.064969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:01.065835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:01.066241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.066959 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:01.070791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:01.071462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:01.071956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.072372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.073300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:01.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:01.074506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.074506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:01.074506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.074506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:01.074947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:01.075665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:01.075665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:01.075665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:01.076945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:01.077352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:01.077530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.078673 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.082568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:01.083148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.083148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:01.083827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:01.084524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:01.085325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:01.085325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:01.085325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:05.284900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:05.284900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:05.285638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:05.286681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:05.325520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:05.325890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.326697 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:05.327845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:05.328326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:05.328794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:05.329282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:05.330112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:05.330112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:05.330112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:05.330112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:05.330112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.330112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:05.991038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:05.991502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:05.991699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:05.992776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:05.993196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:05.993519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.994381 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:05.995946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:05.996427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:05.996866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.996866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:05.997602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:05.998287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:05.999040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:05.999040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:05.999040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:06.030277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:06.030870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:06.031047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:06.031248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:06.031663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:06.031897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.033205 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:06.033699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.034373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.035033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:06.035564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:06.036118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.036118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:06.036118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.036118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:06.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.037109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.037623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:06.064562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:06.064807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:06.066041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:06.066041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:06.066273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:06.066567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.067221 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:06.067885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.068450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.068836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:06.069314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:06.069882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:06.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.072755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:06.075584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:06.075929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:06.076124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:06.076383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:06.076838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:06.077058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:06.077176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.077667 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.078728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.079144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:06.079410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.079410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:06.080107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:06.080464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:06.080841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:06.080841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:06.080841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:10.284805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:10.287377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:10.287377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:10.287377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:10.328455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:10.328635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:10.329683 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:10.331311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:10.331928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:10.332602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:10.333327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:10.334223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:10.334223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:10.334223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:10.334223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:10.334223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:10.334223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:10.991873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:10.992348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.001099 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.001978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.002584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:11.003311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:11.004951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.006469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:11.006469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:11.006469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.006469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:11.006469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.006469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:11.030927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:11.031663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:11.032001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:11.032440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:11.032894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:11.033115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.033923 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:11.035710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.036553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.038047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:11.038833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:11.040239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.040657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.041267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:11.065322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:11.065818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:11.066023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:11.066193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:11.067133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:11.067556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.068399 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:11.069380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.070983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:11.071701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:11.072627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:11.075092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:11.076634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:11.077501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:11.077501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:11.077624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:11.078081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:11.078323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.079028 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.079851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.080183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:11.080440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.080440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:11.080904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:11.081635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:11.082380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:11.082380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:11.082380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:15.286075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:15.286963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:15.287224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:15.288635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:15.324495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:15.324722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.325572 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:15.326481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:15.326909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:15.327282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.327282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:15.327602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:15.328158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:15.328635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:15.328635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.328635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:15.992424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:15.992990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:15.993086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:15.994043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:15.994432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:15.994730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.995743 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:15.997082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:15.997699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:15.998437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.998437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:15.998601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:15.999474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.000622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:16.000622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.000622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:16.031730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:16.032339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:16.032489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:16.032629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:16.033372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:16.033681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.034380 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:16.035115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.035635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.035983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:16.036417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:16.036845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.036845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:16.036845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.036845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:16.037094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.037615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.038165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:16.066152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:16.066846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:16.066978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:16.067133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:16.067686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:16.068107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.068908 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:16.069936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.070392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.071205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:16.071540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:16.071943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.071943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:16.071943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.071943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.072838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:16.073562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:16.074220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.074220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:16.074220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.074220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:16.076805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:16.077255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:16.077414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:16.077651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:16.078103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:16.078358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:16.078505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.079053 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.080330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.080737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:16.081118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.081118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:16.081813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:16.082334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:16.082897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:16.082897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:16.082897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:20.287047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:20.287682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:20.287682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:20.288512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:20.290576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:44:20.327818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:20.328338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:20.331339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:20.331939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:20.332455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:20.333084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:20.333817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:20.333817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:20.333817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:20.333817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:20.333817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.333817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:20.993149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:20.995266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:20.995372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:20.996520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:20.996900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:20.997132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.998055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:20.999064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:20.999369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:20.999640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:20.999640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:21.000047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:21.000425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.000851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:21.000851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.000851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:21.032473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:21.033061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:21.033189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:21.033330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:21.034699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:21.035058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.036243 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:21.037260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.037933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.038320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:21.038718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:21.039306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.039795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.040940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:21.068162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:21.068615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:21.068843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:21.069088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:21.070789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:21.071294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.072178 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:21.073561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.075351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:21.076074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:21.076717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.076717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:21.076717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.076717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.077322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:21.077862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:21.078381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.078381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:21.078381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.078381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:21.079052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:21.079388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:21.079550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:21.079824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:21.080779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:21.081197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:21.081450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.082438 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.083802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.084204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:21.084592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.084592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:21.085195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:21.085741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:21.086283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:21.086283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:21.086283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:25.287645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:25.288266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:25.288266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:25.289087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:25.330587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:25.330841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.331792 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:25.334059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:25.334796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:25.335343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.335343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:25.335911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:25.336506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:25.337102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:25.337102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.337102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:25.995311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:25.995940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:25.996124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:25.996861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:25.997054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:25.997229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.997894 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:25.999216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:25.999822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:26.000223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.000223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:26.000772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:26.001477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.002160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:26.002160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.002160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:26.033102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:26.033702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:26.033854 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:26.033992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:26.034599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:26.034855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.035460 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:26.036709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.037241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.037988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:26.038505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:26.039058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.039058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:26.039058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.039058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:26.039593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.040243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.040731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:26.068911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:26.069495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:26.069635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:26.069843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:26.071312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:26.071590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.072278 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:26.073298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.073853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.074279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:26.074621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:26.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:26.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.075366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:26.075878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:26.076371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.076371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:26.076371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.076371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:26.079995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:26.080517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:26.080652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:26.080877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:26.081545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:26.081810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:26.081955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.082531 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.083785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.084264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:26.084614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.084614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:26.084889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:26.085495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:26.086330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:26.086330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:26.086330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:30.288330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:30.288915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:30.288915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:30.290029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:30.326295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:30.326490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.327072 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:30.328123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:30.328702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:30.329254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.329254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:30.329546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:30.330205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:30.330728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:30.330728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.330728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:30.995917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:30.996395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:30.996522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:30.997380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:30.997603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:30.997754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.998336 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:30.999184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.000002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:31.000607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:31.001435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.002436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:31.002436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:31.002436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.002436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:31.002436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.002436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:31.033719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:31.034236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:31.034426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:31.034647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:31.035349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:31.035658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.036379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:31.037414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.038026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.038488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:31.039012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:31.039567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.039567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:31.039567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.039567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:31.040023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.040636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.041347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:31.069655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:31.070177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:31.070347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:31.070513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:31.071421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:31.072092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.073035 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:31.074022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.074684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.075032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:31.075355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:31.075749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.075749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:31.075749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.075749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.076483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:31.077404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:31.078293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.078293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:31.078293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.078293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:31.080397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:31.080669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:31.080853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:31.081287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:31.081752 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:31.082040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:31.082206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.082911 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.084250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:31.084671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.084671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:31.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:31.085816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:31.086328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:31.086328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:31.086328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:35.289174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:35.289977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:35.290067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:35.291527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:35.337820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:35.337998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.338567 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:35.339848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:35.341127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:35.342079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:35.343397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:35.344399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:35.344399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:35.344399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:35.344399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:35.344399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.344399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:35.996687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:35.997176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:35.997351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:35.998327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:35.998630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:35.998889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:35.999724 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.001285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.001690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:36.002069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.002069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:36.002774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:36.003304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.003887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:36.003887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.003887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:36.034316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:36.034886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:36.035122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:36.035273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:36.035728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:36.035988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.036684 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:36.037968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.038712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.039216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:36.039751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:36.040453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.040453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:36.040453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.040453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:36.040788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.041551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.042523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:36.070312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:36.070810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:36.070953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:36.071046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:36.071632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:36.072009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.072737 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:36.073645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.074277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.074534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:36.074881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:36.075244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.075244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:36.075244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.075244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.075598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:36.076063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:36.076600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.076600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:36.076600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.076600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:36.080796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:36.081040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:36.081100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:36.081238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:36.081608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:36.081908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:36.082098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.082637 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.083404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.083995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:36.084475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.084475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:36.084564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:36.085262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:36.085941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:36.085941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:36.085941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:40.290287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:40.291054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:40.291054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:40.291866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:40.335867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:40.335867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:40.336087 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:40.336851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:40.337278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:40.337806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:40.337806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:40.337980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:40.338493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:40.339148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:40.339148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:40.339148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:40.997278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:40.998224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:40.998340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:41.001343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:41.001343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:41.001343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.001855 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.003090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.004033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:41.005158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:41.005999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.007026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:41.007026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:41.007026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.007026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:41.007026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.007026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:41.034947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:41.035716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:41.035938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:41.036137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:41.037792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:41.037792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.038089 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:41.039867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.040399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.040884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:41.041647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:41.042312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.042312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:41.042312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.042312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:41.042488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.043152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.044173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:41.070891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:41.071220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:41.071324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:41.071467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:41.072537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:41.072537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.072647 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:41.073436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.073935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.074379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:41.074725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:41.075048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.075048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:41.075048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.075048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.075479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:41.075968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:41.076348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.076348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:41.076348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.076348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:41.081464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:41.081655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:41.081727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:41.081975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:41.082459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:41.082891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:41.083204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.083878 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.084831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.085256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:41.085692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.085692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:41.085873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:41.086558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:41.087261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:41.087261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:41.087261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:45.291003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:45.292126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:45.292126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:45.292578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:45.340636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:45.341134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:45.342072 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:45.343217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:45.343937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:45.344624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:45.345461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:45.346610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:45.346610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:45.346610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:45.346610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:45.346610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:45.346610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:45.998128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:45.998429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:45.998592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:45.999947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:46.000342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:46.000592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.001574 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.003313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.003659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:46.003997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.003997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:46.005112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:46.005518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.006049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:46.006049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.006049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:46.035504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:46.036020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:46.036245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:46.036380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:46.036878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:46.037176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.037701 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:46.038883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.039534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.040322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:46.041410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:46.041786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.042972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.043494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:46.071499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:46.072011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:46.072126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:46.072700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:46.073081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.073703 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:46.074655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.075049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.075577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:46.075910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:46.076244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.076244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:46.076244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.076244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.076657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:46.077122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:46.077568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.077568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:46.077568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.077568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:46.081795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:46.082147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:46.082308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:46.082447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:46.082976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:46.083129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:46.083268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.083913 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.084937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.085261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:46.085551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.085551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:46.085930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:46.086325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:46.086815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:46.086815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:46.086815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:50.291507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:50.292207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:50.292207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:50.292937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:50.329608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:50.329870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:50.330427 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:50.331506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:50.332129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:50.332667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:50.333312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:50.333917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:50.333917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:50.333917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:50.333917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:50.333917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:50.333917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:50.998590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:50.999125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:50.999208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:51.000084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:51.000270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:51.000465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.001186 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.001941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.002468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:51.003098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:51.003641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.004362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:51.004362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:51.004362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.004362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:51.004362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.004362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:51.036056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:51.036566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:51.036803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:51.036990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:51.037662 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:51.038063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.038713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:51.040057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.041002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.042126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:51.042685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:51.043056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.043443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.043917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:51.072261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:51.072830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:51.072983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:51.073187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:51.073809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:51.074149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:51.075718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.076027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.077348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:51.077706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:51.078095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.078095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:51.078095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.078095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.078788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:51.079384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:51.079972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.079972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:51.079972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.079972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:51.082311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:51.082663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:51.082743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:51.082921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:51.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:51.084229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:51.084660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.085624 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.086510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.086916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:51.087418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.087418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:51.087509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:51.088023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:51.088570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:51.088570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:51.088570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:55.292165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:55.293425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:55.293678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:55.294998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:55.327922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:55.328426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:55.329498 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:55.330318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:55.330753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:55.331200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:55.331200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:55.331555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:55.332276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:55.332978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:55.332978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:55.332978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:55.999114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:55.999719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:55.999973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:56.000873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:56.001154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:56.001335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.001987 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.003024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.003628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:56.004271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:56.005061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.005973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:56.005973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:56.005973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.005973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:56.005973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.005973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:56.036951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:56.038044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:56.038248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:56.038462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:44:56.039107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:56.039437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.040416 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:56.042072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.042747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.043776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:56.044293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:56.044874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.044874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:56.044874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.044874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:56.046049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.046693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.047307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:56.072908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:56.073371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:56.073565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:56.073775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:56.074205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:56.074464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.075070 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:44:56.076277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.076881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.077935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:56.078417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:56.078966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.078966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:56.078966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.078966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.079605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:56.080274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:56.081028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.081028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:56.081028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.081028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:44:56.082720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:44:56.082949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:44:56.083047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:44:56.083217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:44:56.083542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:56.083834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:44:56.083988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.084430 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.085203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.085750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:44:56.086164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.086164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:44:56.086213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:44:56.086534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:44:56.086973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:44:56.086973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:44:56.086973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:00.293576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:00.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:00.294217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:00.295075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:00.331424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:00.331785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:00.332563 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:00.333487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:00.333970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:00.334390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:00.334975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:00.335615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:00.335615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:00.335615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:00.335615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:00.335615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:00.335615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:00.999741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:01.000264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:01.000375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:01.001332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:01.001548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:01.001669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.002288 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.003094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.003726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:01.004281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:01.004901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.005670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:01.005670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:01.005670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.005670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:01.005670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.005670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:01.038051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:01.038654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:01.038820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:01.038906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:01.039968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:01.040300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.041415 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:01.042368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:01.043400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.044219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.044925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:01.045711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.046620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.047261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:01.073479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:01.073990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:01.074182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:01.074468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:01.075201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:01.075621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.076456 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:01.077668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.079027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:01.079595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:01.080261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.080261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:01.080261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.080261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.080457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:01.081305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:01.082220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.082220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:01.082220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.082220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:01.083263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:01.083544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:01.083651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:01.083858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:01.084273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:01.084540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:01.084711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.085398 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.086277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.086687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:01.087067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.087067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:01.087291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:01.087867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:01.088366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:01.088366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:01.088366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:05.294423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:05.295372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:05.295475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:05.296865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:05.331150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:05.331628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:05.332573 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:05.333479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:05.334137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:05.334869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:05.335700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:05.336825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:05.336825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:05.336825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:05.336825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:05.336825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:05.336825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:06.000499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:06.000938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:06.001116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:06.002286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:06.002735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:06.003100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.004005 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.005083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.005716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:06.006171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.006171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:06.006288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:06.006736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.007320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:06.007320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.007320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:06.038805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:06.039486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:06.039779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:06.040159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:06.040781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:06.041106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.042136 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:06.043072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.043721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.044006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:06.044368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:06.044724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.045143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.045646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:06.074149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:06.074890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:06.075198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:06.075524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:06.076321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:06.076792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.077679 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:06.078490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.079195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.079629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:06.081148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:06.082252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:06.083111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:06.083918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:06.084407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:06.084407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:06.084407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:06.084617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:06.085073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:06.085331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.086034 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.086861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.087403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:06.087978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.087978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:06.088101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:06.088790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:06.089535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:06.089535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:06.089535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:10.295220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:10.296194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:10.296194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:10.297657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:10.334873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:10.335180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:10.336047 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:10.337049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:10.337745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:10.338429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:10.339242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:10.340136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:10.340136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:10.340136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:10.340136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:10.340136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:10.340136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:11.001179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:11.001603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:11.001746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:11.002670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:11.003124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:11.003361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.004565 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.005367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.005922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:11.006339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:11.006978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.007796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:11.007796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:11.007796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.007796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:11.007796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.007796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:11.039512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:11.040116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:11.040271 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:11.040422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:11.040971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:11.041224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.041902 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:11.042943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.043458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.043880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:11.044186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:11.044437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.044437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:11.044437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.044437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:11.045494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.046012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.046555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:11.074685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:11.075172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:11.075333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:11.075522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:11.076110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:11.076497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.077419 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:11.078529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.079205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.079710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:11.080242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:11.080792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.080792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:11.080792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.080792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.081125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:11.081864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:11.082597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.082597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:11.082597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.082597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:11.084187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:11.084515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:11.084673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:11.084908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:11.085372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:11.085635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:11.085820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.086424 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.087407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.087851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:11.088162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.088162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:11.088542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:11.088931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:11.089396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:11.089396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:11.089396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:15.295779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:15.296440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:15.296504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:15.297806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:15.332932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:15.335885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:15.336575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:15.337057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:15.337518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:15.338274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:16.002566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:16.003259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:16.003465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:16.004463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:16.004690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:16.004841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.005413 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.006982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.007329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:16.007647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.007647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:16.008708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:16.009077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.009434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:16.009434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.009434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:16.040155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:16.040418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:16.040539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:16.040671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:16.041324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:16.041496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.042109 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:16.042935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.043462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.043702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:16.044109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:16.044538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.044538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:16.044538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.044538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:16.044703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.045395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.046121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:16.075220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:16.077350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:16.077350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:16.077350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:16.077350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:16.077350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.077860 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:16.079072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.079901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.080654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:16.081172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:16.081724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.081724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:16.081724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.081724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.082402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:16.083175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:16.083973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.083973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:16.083973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.083973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:16.084778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:16.085217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:16.085379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:16.085577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:16.085909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:16.086106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:16.086203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.086713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.087667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.088240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:16.088785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.088785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:16.088883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:16.089539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:16.090308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:16.090308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:16.090308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:20.296424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:20.297647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:20.297780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:20.298564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:20.300638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:45:20.342527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:20.342852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:20.343540 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:20.344627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:20.345286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:20.345945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:20.346652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:20.347568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:20.347568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:20.347568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:20.347568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:20.347568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:20.347568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:21.003486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:21.003921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:21.004156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:21.005416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:21.005830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:21.006089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.006984 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.008828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.009519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:21.010284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.010284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:21.011044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:21.011996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.013002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:21.013002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.013002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:21.040842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:21.041376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:21.041597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:21.041848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:21.042627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:21.042968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.043823 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:21.045367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.045963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.046989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:21.047551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:21.048157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.048157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:21.048157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.048157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:21.049091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.049856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.050696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:21.075862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:21.076335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:21.076660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:21.077020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:21.078087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:21.078516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.079342 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:21.080042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:21.080931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:21.081558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:21.082201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:21.082822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.083708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:21.085188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:21.085403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:21.085483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:21.085646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:21.086160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:21.086423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:21.086586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.087214 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.088224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.088691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:21.089087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.089087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:21.089779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:21.090213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:21.090697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:21.090697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:21.090697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:25.297905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:25.299080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:25.299080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:25.299928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:25.336412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:25.336721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:25.337245 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:25.338486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:25.339108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:25.339722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:25.339722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:25.339984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:25.340561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:25.341165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:25.341165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:25.341165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:26.004078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:26.004706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:26.005123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:26.006124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:26.006456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:26.006692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.007686 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.009350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.009734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:26.010089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.010089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:26.011269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:26.012318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.012804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:26.012804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.012804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:26.041621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:26.042260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:26.042364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:26.042521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:26.043305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:26.043715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.044678 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:26.046174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.047019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.047627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:26.048214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:26.048823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.048823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:26.048823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.048823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:26.049559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.050332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.050973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:26.076411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:26.076979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:26.077148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:26.077313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:26.078213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:26.078799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.079601 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:26.080493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.081195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.081643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:26.082395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:26.083128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.083128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:26.083128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.083128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.083555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:26.084445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:26.085255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.085255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:26.085255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.085255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:26.085480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:26.085852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:26.086057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:26.086279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:26.086886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:26.087123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:26.087990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.088463 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.089904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:26.090448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:26.091060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:26.091887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:26.091887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:26.091887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:26.091887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:26.091887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:26.091887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:30.297501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:30.298535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:30.298535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:30.298535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:30.308132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 01:45:30.308364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 01:45:30.308496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:45:30.342330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.342586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.342974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.343206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.343880 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.344946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.345823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.346871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.346871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.346871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.346871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:45:30.379581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.379894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.380266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.380266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.380880 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.381977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.382368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.382736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.382736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.382736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.382736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:45:30.417701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.417701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.418332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.418532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.419958 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.421829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.422074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.422496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.422496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.422496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.422496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:45:30.450357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.450357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.450612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.450692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.451162 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.452037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.452460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.452910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.452910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.452910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.452910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:45:30.543913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.544196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.544308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.544538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.544660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.546994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 01:45:30.547744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.548449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.549348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:45:30.654645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.654886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.655453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.655946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.656922 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.658165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.659217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.360 seconds
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:30.659960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:30.691400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:30.691796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.692716 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.693481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.694176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:30.694639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:30.695236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:30.695862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:30.695862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:30.695862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:30.695862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:30.695862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:30.695862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:31.004682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:31.005315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:31.005511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:31.006533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:31.007033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:31.007300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.008294 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.009303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.009672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:31.010023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.010023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:31.010477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:31.011114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.011689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:31.011689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.011689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:31.043018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:31.043606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:31.043910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:31.044179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:31.044886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:31.045290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.046345 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:31.047957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.048504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.048876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:31.049257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:31.049723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.049723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:31.049723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.049723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:31.049990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.050541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.051026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:31.076889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:31.077468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:31.077730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:31.078056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:31.078476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:31.078724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.079381 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:31.080541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.081026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.082000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:31.082357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:31.082794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.082794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:31.082794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.082794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.083746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:31.084298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:31.084785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.084785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:31.084785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.084785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:31.085876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:31.086193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:31.086302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:31.086470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:31.086812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:31.087077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:31.087266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.087813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.088953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:31.089281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.089281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:31.089569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:31.090066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:31.090667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:31.090667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:31.090667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:35.658867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:35.659742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:35.659742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:35.660977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:35.699336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:35.699732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:35.700634 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:35.701927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:35.702646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:35.703261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:35.703261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:35.703872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:35.705241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:35.705874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:35.705874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:35.705874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:36.005268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:36.005902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:36.006143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:36.007135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:36.007414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:36.007611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.008172 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.009041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.009584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:36.010188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:36.010882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.011621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:36.011621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:36.011621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.011621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:36.011621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.011621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:36.043520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:36.043968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:36.044047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:36.044191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:36.045063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:36.045298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.046133 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:36.046967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.047646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.048063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:36.048622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:36.049314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.050089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.051301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:36.077465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:36.078105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:36.078304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:36.078518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:36.078930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:36.079136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.079770 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:36.081220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.081789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.082780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:36.083166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:36.083536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.083536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:36.083536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.083536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.084620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:36.085173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:36.085678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.085678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:36.085678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.085678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:36.086253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:36.086608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:36.086824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:36.087103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:36.087555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:36.087935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:36.088122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.088782 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.089748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.090159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:36.090525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.090525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:36.090904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:36.091385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:36.092089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:36.092089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:36.092089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:40.659394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:40.660445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:40.660445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:40.661630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:40.693325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:40.693718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:40.694510 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:40.695481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:40.696495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:40.697459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:40.698632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:40.700412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:40.700412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:40.700412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:40.700412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:40.700412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:40.700412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:41.005958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:41.006310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:41.006433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:41.007371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:41.007627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:41.007812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.008403 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.009392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.009971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:41.010387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:41.010904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.011502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:41.011502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:41.011502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.011502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:41.011502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.011502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:41.044106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:41.055706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:41.055876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:41.055964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:41.056710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:41.057079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.058008 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:41.059396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.060037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.060438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:41.060725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:41.061087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.061087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:41.061087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.061087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:41.061527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.061856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.062275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:41.078173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:41.078722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:41.078932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:41.079145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:41.079908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:41.080387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.081372 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:41.082321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.082779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.083371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:41.083739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:41.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:41.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.084887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:41.085487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:41.086036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.086036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:41.086036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.086036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:41.086715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:41.087103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:41.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:41.087485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:41.088230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:41.088537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:41.088716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.089331 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.090277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.090659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:41.091046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.091046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:41.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:41.091882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:41.092468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:41.092468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:41.092468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:45.660072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:45.661115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:45.661115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:45.662876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:45.705303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:45.705519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:45.706429 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:45.708606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:45.709478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:45.709979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:45.709979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:45.710668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:45.711340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:45.711859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:45.711859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:45.711859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:46.006547 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:46.007083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:46.007211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:46.008420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:46.008831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:46.009189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.010061 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.011240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.011819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:46.012415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.012415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:46.012697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:46.013562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.014161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:46.014161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.014161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:46.056147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:46.056671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:46.056822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:46.056980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:46.057701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:46.058088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.058888 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:46.059691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.060182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.060475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:46.060889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:46.061256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.061256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:46.061256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.061256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:46.061477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.061894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.062383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:46.078663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:46.079207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:46.079322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:46.079468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:46.080147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:46.080564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.081389 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:46.082268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.082697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.083152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:46.083532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:46.083965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.083965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:46.083965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.083965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.084253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:46.084935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:46.085542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.085542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:46.085542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.085542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:46.087199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:46.087519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:46.087636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:46.087847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:46.088434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:46.088887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:46.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.089948 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.090833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:46.091500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.091500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:46.091806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:46.092144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:46.092590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:46.092590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:46.092590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:50.660925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:50.661739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:50.661739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:50.663019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:50.699639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:50.699887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:50.700666 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:50.701555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:50.702025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:50.702431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:50.703014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:50.703435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:50.703435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:50.703435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:50.703435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:50.703435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:50.703435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:51.007210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:51.007795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:51.007981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:51.008869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:51.009117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:51.009258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.009957 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.011712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.012344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:51.012921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.012921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:51.013774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:51.014508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.015023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:51.015023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.015023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:51.056824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:51.057342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:51.057477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:51.057604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:51.058264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:51.058470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.059097 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:51.059993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.060565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.060914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:51.061181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:51.061438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.061438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:51.061438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.061438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:51.061917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.062280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.062692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:51.079339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:51.080291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:51.080452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:51.080605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:51.081191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:51.081512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.082218 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:51.083954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.084976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.085581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:51.086157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:51.086729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.086729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:51.086729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.086729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.087325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:51.088008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:51.089089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:51.089089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:51.089186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.089685 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.090549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.091121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:51.091588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.091588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:51.091670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:51.092314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:51.093032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:51.093032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:51.093032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:55.661682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:55.662459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:55.662459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:55.663172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:55.699718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:55.699957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:55.700589 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:55.701489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:55.702087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:55.702702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:55.703373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:55.704209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:55.704209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:55.704209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:55.704209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:55.704209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:55.704209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:56.007802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:56.008410 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:56.008608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:56.009415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:56.009595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:56.009783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.010415 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.011846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.012635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:56.013417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:56.014332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.015175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:56.015175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:56.015175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.015175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:56.015175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.015175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:56.057520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:56.057950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:56.058156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:56.058359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:45:56.058900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:56.059212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.059978 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:56.063687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.064384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.065143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:56.065741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:56.066557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.066557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:56.066557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.066557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:56.066825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.067504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.068308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:56.079857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:56.080185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:56.080382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:56.080583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:56.080980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:56.081281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.081911 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:45:56.083042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.083577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.084574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:56.084919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:56.085452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.085452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:56.085452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.085452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.086281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:56.087175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:56.088013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.088013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:56.088013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.088013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:45:56.088324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:45:56.088648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:45:56.088830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:45:56.089061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:45:56.089446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:56.089649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:45:56.089803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.090281 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.091189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.092033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:45:56.092900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:45:56.094956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:45:56.096445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:45:56.096445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:45:56.096445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:45:56.096445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:45:56.096445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:45:56.096445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:00.662278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:00.663228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:00.663228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:00.664605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:00.711185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:00.711538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:00.712499 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:00.713533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:00.714397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:00.715607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:00.716330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:00.716996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:00.716996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:00.716996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:00.716996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:00.716996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:00.716996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:01.008594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:01.009597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:01.010043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:01.011306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:01.011738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:01.012008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.013093 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.014545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.015115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:01.015595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.015595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:01.016932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:01.018881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.020232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:01.020232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.020232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:01.058112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:01.058775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:01.058947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:01.059092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:01.059864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:01.060248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.061410 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:01.063081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.063952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.065558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:01.067495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:01.068356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.069025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.070929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:01.080323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:01.081147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:01.081397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:01.081698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:01.082937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:01.082937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.086147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:01.086853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.087631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:01.088092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:01.088650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:01.089320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:01.089694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:01.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:01.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:01.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.091376 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.092348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.093263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:01.094331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:01.095673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:01.096966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:01.096966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:01.096966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:01.096966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:01.096966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:01.096966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:05.663143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:05.664015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:05.664015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:05.664386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:05.695160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:05.696609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:05.696945 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:05.697737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:05.698488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:05.699485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:05.700825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:05.702310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:05.702310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:05.702310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:05.702310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:05.702310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:05.702310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:06.009529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:06.009986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:06.010155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:06.011929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:06.011929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:06.011929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.012944 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.014298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.015167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:06.016117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.016117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:06.017617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:06.018212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.018806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:06.018806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.018806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:06.058986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:06.059894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:06.060156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:06.060369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:06.061045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:06.061398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.062311 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:06.063779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.064282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.065020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:06.065479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:06.065910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.065910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:06.065910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.065910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:06.066613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.067086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:06.081021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:06.081466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:06.081645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:06.081888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:06.082467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:06.082855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.083701 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:06.084929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.085632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.086126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:06.086689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:06.087487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:06.088493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.089226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:06.089620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:06.090060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:06.090230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:06.090414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:06.090897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:06.091232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:06.091505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.092236 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.093482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.093985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:06.094466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.094466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:06.094877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:06.095340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:06.095901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:06.095901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:06.095901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:10.663740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:10.664591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:10.664591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:10.665541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:10.709222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:10.709605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:10.710390 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:10.711368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:10.711798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:10.712210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:10.712210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:10.712577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:10.713372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:10.714349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:10.714349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:10.714349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:11.010231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:11.011003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:11.011152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:11.012327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:11.012647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:11.013190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.014303 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.015514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.016009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:11.016551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:11.017327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.018024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:11.018024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:11.018024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.018024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:11.018024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.018024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:11.059948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:11.060676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:11.060987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:11.061274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:11.062098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:11.062460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.063443 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:11.064815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.065941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.066184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:11.066537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:11.066942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.066942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:11.066942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.066942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:11.067404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.067881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.068349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:11.081996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:11.082375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:11.082492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:11.082634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:11.083368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:11.083835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.084608 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:11.085608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.086393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.086876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:11.087453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:11.088235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:11.089372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.090122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:11.090477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:11.090477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:11.090600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:11.090787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:11.091258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:11.091732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:11.091918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.093643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.094195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:11.095041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:11.095599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:11.096286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:11.096286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:11.096286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:11.096286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:11.096286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:11.096286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:15.664507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:15.665170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:15.665170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:15.665829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:15.712023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:15.712422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:15.713257 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:15.714068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:15.714557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:15.715374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:15.716228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:15.716934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:15.716934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:15.716934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:15.716934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:15.716934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:15.716934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:16.010916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:16.011488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:16.011682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:16.012855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:16.013157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:16.013319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.014019 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.016087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.016462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:16.016832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.016832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:16.017281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:16.017705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.018184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:16.018184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.018184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:16.060637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:16.061273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:16.061321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:16.061522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:16.062145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:16.062472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.063293 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:16.064567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.065549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.066078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:16.066810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:16.067526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.067526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:16.067526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.067526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:16.068052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.068529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.071336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:16.082552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:16.083155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:16.083290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:16.083432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:16.084011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:16.084390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.085138 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:16.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.086918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.087363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:16.087979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:16.088661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:16.089325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.090300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:16.090896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:16.091174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:16.091363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:16.091592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:16.092100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:16.092454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:16.092673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.093377 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.094377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.094818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:16.095178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.095178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:16.095450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:16.095925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:16.096473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:16.096473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:16.096473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:20.665227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:20.666277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:20.666516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:20.667900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:20.669867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=434 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:46:20.705442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:20.705855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:20.706856 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:20.707678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:20.708122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:20.708496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:20.709685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:20.711612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:20.711612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:20.711612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:20.711612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:20.711612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:20.711612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:21.011615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:21.012035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:21.012233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:21.013529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:21.013908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:21.014207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.015071 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.016307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.016960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:21.017339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.017339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:21.018039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:21.018828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.019433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:21.019433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.019433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:21.061264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:21.061616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:21.061824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:21.062017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:21.062832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:21.063126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.063955 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:21.065330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.065861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.066693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:21.067101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:21.067470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.067470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:21.067470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.067470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:21.068403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.069852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.070711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:21.083310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:21.083652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:21.083806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:21.083899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:21.084706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:21.085156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.086222 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:21.087128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.087528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.088024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:21.088351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:21.088753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.088753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:21.088753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.088753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.089183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:21.089697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:21.090230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.090230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:21.090230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.090230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:21.091164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:21.091379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:21.091520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:21.091735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:21.092265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:21.092685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:21.092973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.093840 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.095717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.096201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:21.096566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.096566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:21.096956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:21.097421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:21.097942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:21.097942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:21.097942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:25.666600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:25.667506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:25.667506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:25.669052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:25.706797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:25.707306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:25.708289 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:25.709028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:25.709955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:25.710639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:25.711094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:25.711629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:25.711629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:25.711629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:25.711629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:25.711629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:25.711629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:26.012075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:26.012356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:26.012612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:26.013458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:26.013869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:26.014233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.015254 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.016031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.016822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:26.017347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.017347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:26.017906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:26.018487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.019014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:26.019014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.019014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:26.062050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:26.062665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:26.062913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:26.063126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:26.063945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:26.064288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.065203 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:26.066799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.067660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.068343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:26.069001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:26.069672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.069672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:26.069672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.069672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:26.070429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.070886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.071399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:26.083893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:26.084347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:26.084549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:26.084845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:26.085603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:26.086266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.087000 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:26.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.088722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.089782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:26.090154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:26.090528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.090528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:26.090528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.090528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.091307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:26.091747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:26.092267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:26.092627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:26.092734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:26.092997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:26.093188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.093717 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.094634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.095238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:26.095890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:26.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:26.097625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:26.097625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:26.097625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:26.097625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:26.097625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:26.097625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:30.667415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:30.668083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:30.668254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:30.669247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:30.709578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:30.710218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:30.711274 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:30.712356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:30.712744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:30.713149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:30.713149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:30.713439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:30.713942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:30.714447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:30.714447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:30.714447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:31.012867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:31.013461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:31.013685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:31.015206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:31.015612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:31.015940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.016981 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.018281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.018994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:31.019635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.019635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:31.019856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:31.020631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.021213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:31.021213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.021213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:31.062871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:31.063545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:31.063682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:31.063853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:31.064395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:31.064704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.065497 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:31.066249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.067043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.067343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:31.067906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:31.068344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.068822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.069557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:31.085048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:31.085131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:31.085244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:31.085777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:31.086193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.086936 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:31.087444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.088054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.088435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:31.088924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:31.089460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:31.090019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.090747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:31.092284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:31.092708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:31.093052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:31.093442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:31.093952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:31.094329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:31.094519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.095031 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.095825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.096415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:31.097309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:31.098044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:31.098993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:31.098993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:31.098993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:31.098993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:31.098993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:31.098993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:35.668185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:35.669517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:35.669517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:35.670194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:35.707827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:35.707827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:35.708055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:35.708766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:35.709233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:35.709752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:35.709752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:35.709949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:35.710371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:35.710800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:35.710800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:35.710800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:36.013889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:36.014150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:36.014467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:36.015479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.017507 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.019428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.019872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:36.020276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.020276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:36.021704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:36.022350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.023113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:36.023113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.023113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:36.063589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:36.064064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:36.064270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:36.064476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:36.065860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:36.065860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.066167 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:36.067562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.068204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.069053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:36.069455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:36.069888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.069888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:36.069888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.069888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:36.070898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.071433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.072236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:36.085108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:36.085580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:36.085799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:36.086112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:36.086887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:36.087139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.087791 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:36.089219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.090846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:36.091475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:36.091922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.091922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:36.091922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.091922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.092773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:36.093312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:36.093849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:36.094495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:36.095019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:36.095331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.096327 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.097355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:36.097736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:36.098140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:36.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:36.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:36.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:36.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:36.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:36.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:40.669256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:40.670222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:40.670222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:40.670874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:40.707996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:40.708411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:40.709539 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:40.710797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:40.711599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:40.712510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:40.714025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:40.715093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:40.715093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:40.715093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:40.715093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:40.715093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:40.715093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:41.014427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:41.014736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:41.014886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:41.015621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:41.015902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:41.016075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.016644 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.017604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.018554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:41.019203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:41.020040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.021257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:41.021257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:41.021257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.021257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:41.021257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.021257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:41.064447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:41.065186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:41.065375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:41.065640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:41.066954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:41.066954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.067371 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:41.068940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.070018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.070685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:41.071423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:41.072044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.072740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.073675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:41.085791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:41.086200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:41.086350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:41.086495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:41.087079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:41.087577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.088540 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:41.089370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.090147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:41.091504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:41.092355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:41.093406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:41.094734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:41.095288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:41.095288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:41.095288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:41.095514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:41.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.096157 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.097040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.097573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:41.098176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.098176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:41.098340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:41.098957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:41.099438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:41.099438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:41.099438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:45.670250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:45.671113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:45.671398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:45.672636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:45.711205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:45.711432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:45.712260 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:45.715812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:45.716306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:45.716701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:45.717214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:45.717725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:45.717725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:45.717725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:45.717725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:45.717725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:45.717725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:46.015472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:46.016039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:46.016107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:46.017211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:46.017448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:46.017611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.018300 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.019871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.020430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:46.020930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.020930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:46.021302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:46.021915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.022539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:46.022539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.022539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:46.065240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:46.065749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:46.065891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:46.066029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:46.066842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:46.067130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.068233 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:46.068813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.070157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:46.070868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:46.071535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.072275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.073152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:46.086351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:46.086740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:46.086969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:46.087190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:46.088025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:46.088514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.089407 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:46.091074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.091793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.092947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:46.093544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:46.094277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:46.094980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:46.095934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:46.096701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:46.096701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:46.096701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:46.096701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:46.096701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.097006 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.097902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.098299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:46.098667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.098667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:46.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:46.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:46.100130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:46.100130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:46.100130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:50.670979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:50.672287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:50.672287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:50.673052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:50.707569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:50.707949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:50.708361 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:50.709361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:50.709987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:50.711199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:50.711728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:50.712303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:50.712303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:50.712303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:50.712303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:50.712303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:50.712303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:51.016118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:51.016598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:51.016802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:51.017898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:51.018178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:51.018384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.019271 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.020715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.021370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:51.022699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:51.024186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.025462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:51.025462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:51.025462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.025462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:51.025462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.025462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:51.065899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:51.066295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:51.066402 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:51.066531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:51.066900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:51.067100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.067572 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:51.068419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.069239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.069649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:51.070239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:51.070917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.073482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:51.087026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:51.087406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:51.087574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:51.087776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:51.088301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:51.088665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.089436 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:51.090779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.091910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.092563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:51.093389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:51.094387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.094387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:51.094387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.094387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.094776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:51.095241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:51.095942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:51.096329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:51.096329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:51.096329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.096813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.097799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.098446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:51.098998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:51.099565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:51.100164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:51.100164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:51.100164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:51.100164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:51.100164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:51.100164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:55.672020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:55.673030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:55.673030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:55.674254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:55.711858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:55.712368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:55.713335 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:55.714920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:55.715591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:55.716364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:55.717298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:55.718296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:55.718296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:55.718296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:55.718296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:55.718296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:55.718296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:56.016926 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:56.017288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:56.017413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:56.018163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:56.018388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:56.018692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.019658 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.021242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.021901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:56.022467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.022467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:56.023709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:56.024468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.025303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:56.025303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.025303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:56.066635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:56.067324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:56.067503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:56.067682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:46:56.068311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:56.068711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.069604 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:56.070570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:56.071629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.072533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.073339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:56.074192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.075296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.075966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:56.087664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:56.088172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:56.088294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:56.088532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:56.088904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:56.089445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.089721 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:46:56.090471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.091117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.091463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:56.092133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:56.092656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:56.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.093993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:46:56.095534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:46:56.095832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:46:56.095933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:46:56.096093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:46:56.096568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:56.096967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:46:56.097196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.097953 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.098615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.099195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:46:56.099676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:46:56.100305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:46:56.101025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:46:56.101025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:46:56.101025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:46:56.101025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:46:56.101025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:46:56.101025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:00.672663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:00.673623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:00.673623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:00.674014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:00.708453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:00.708678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:00.709323 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:00.710413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:00.710885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:00.711310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:00.711310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:00.711684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:00.712288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:00.712867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:00.712867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:00.712867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:01.017691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:01.017976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:01.017976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:01.019038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:01.019431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:01.019791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.020692 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.021563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.022311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:01.023026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:01.023683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.024543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:01.024543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:01.024543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.024543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:01.024543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.024543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:01.067400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:01.067978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:01.068203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:01.068412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:01.069098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:01.069487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.070414 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:01.071561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:01.072718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.073553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.074232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:01.075056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.075745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.076292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:01.088270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:01.088736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:01.089040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:01.089343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:01.090003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:01.090522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.091503 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:01.093171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.094197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.095347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:01.095989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:01.096555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.096555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:01.096555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.096555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.097105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:01.097890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:01.098478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:01.098738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:01.098738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:01.098738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.099266 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.100043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.100414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:01.100838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.100838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:01.101021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:01.101358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:01.101721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:01.101721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:01.101721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:05.673269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:05.674209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:05.674209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:05.675700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:05.718436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:05.718752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:05.719683 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:05.720928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:05.721666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:05.722567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:05.723576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:05.725914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:05.725914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:05.725914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:05.725914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:05.725914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:05.725914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:06.018220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:06.018559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:06.018559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:06.019322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:06.019540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:06.019731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.020485 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.021459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.021889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:06.022486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:06.023048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.023775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:06.023775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:06.023775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.023775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:06.023775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.023775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:06.068143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:06.068486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:06.068486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:06.068712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:06.070467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:06.070810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.071777 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:06.073655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.075282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.075847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:06.076479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:06.077665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.078413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.079341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:06.088878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:06.089167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:06.089293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:06.089509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:06.090183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:06.090600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.091403 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:06.092238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.092684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.093170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:06.093609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:06.094152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.094152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:06.094152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.094152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.094308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:06.094999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:06.095650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.095650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:06.095650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.095650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:06.097732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:06.098145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:06.098339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:06.098623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:06.098942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:06.099172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:06.099322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.099962 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.101510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.101902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:06.102239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.102239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:06.102695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:06.103171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:06.103684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:06.103684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:06.103684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:10.674011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:10.674695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:10.674801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:10.676027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:10.708303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:10.708519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:10.709535 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:10.710158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:10.710911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:10.711928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:10.712494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:10.713291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:10.713291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:10.713291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:10.713291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:10.713291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:10.713291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:11.018605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:11.019098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:11.019215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:11.020165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:11.020482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:11.020729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.021592 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.022612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.023147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:11.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:11.024589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.025522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:11.025522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:11.025522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.025522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:11.025522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.025522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:11.068716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:11.069422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:11.069583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:11.069777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:11.070422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:11.070787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.071653 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:11.072505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.073185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.073624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:11.074178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:11.074728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.075354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.076315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:11.089225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:11.089781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:11.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:11.090166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:11.090787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:11.091278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.092361 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:11.094005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.095045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.096261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:11.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:11.098084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.098084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:11.098084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.098084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.099007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:11.099686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:11.100348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:11.100655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:11.100655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:11.100754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.101274 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.102231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.102656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:11.103412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.103412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:11.103545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:11.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:11.104850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:11.104850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:11.104850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:15.674676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:15.675341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:15.675495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:15.676450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:15.713461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:15.713648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:15.714278 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:15.715328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:15.716258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:15.716851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:15.717518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:15.718352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:15.718352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:15.718352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:15.718352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:15.718352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:15.718352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:16.019445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:16.020008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:16.020226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:16.021442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:16.022391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:16.022391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.023547 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.025595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.026203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:16.026688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.026688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:16.028724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:16.029399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.030134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:16.030134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.030134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:16.069298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:16.069606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:16.069716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:16.069893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:16.070447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:16.070667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.071319 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:16.072303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.073141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.073605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:16.074458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:16.075159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.075556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.076151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:16.089614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:16.089934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:16.090048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:16.090132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:16.090628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:16.090921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.091782 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:16.093099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.094074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.094657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:16.095250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:16.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:16.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.096334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:16.097076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:16.097832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.097832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:16.097832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.097832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:16.099622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:16.099870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:16.100001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:16.100178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:16.100505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:16.100740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:16.100975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.101576 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.102509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.103005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:16.103417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.103417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:16.103603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:16.104096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:16.104612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:16.104612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:16.104612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:20.675531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:20.676417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:20.676417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:20.677396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:20.679129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:47:20.723325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:20.723675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:20.724888 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:20.726103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:20.726962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:20.727511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:20.728228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:20.729010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:20.729010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:20.729010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:20.729010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:20.729010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:20.729010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:21.020398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:21.021644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:21.021644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:21.023171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:21.023844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:21.024226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.025514 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.027427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.027871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:21.028267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.028267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:21.029400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:21.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.030483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:21.030483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.030483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:21.069912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:21.070385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:21.070510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:21.070790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:21.071337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:21.071589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.072355 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:21.073231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.073731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.074217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:21.074583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:21.075038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.075038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:21.075038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.075038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:21.075371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.075892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.076388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:21.090227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:21.090865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:21.091066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:21.091346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:21.091977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:21.092188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.093143 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:21.094496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.094996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.096301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:21.096661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:21.097173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.097173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:21.097173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.097173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.098211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:21.098840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:21.099391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.099391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:21.099391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.099391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:21.100083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:21.100519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:21.100657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:21.100884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:21.101383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:21.101852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:21.102085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.102840 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.103869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.104243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:21.104636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.104636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:21.105117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:21.105783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:21.106374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:21.106374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:21.106374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:25.676403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:25.677213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:25.677315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:25.678617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:25.710563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:25.710778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:25.711193 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:25.711860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:25.712378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:25.712882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:25.713493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:25.714251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:25.714251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:25.714251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:25.714251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:25.714251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:25.714251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:26.020973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:26.021365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:26.021561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:26.022683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:26.023002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:26.023204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.024070 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.025701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.026514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:26.027119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.027119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:26.027610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:26.028417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.029331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:26.029331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.029331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:26.070727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:26.071042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:26.071140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:26.071289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:26.072126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:26.072367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.073158 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:26.074523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.075332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.076039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:26.076671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:26.077405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.077405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:26.077405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.077405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:26.077988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.078433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.078867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:26.090651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:26.090850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:26.091078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:26.091078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:26.091739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:26.092246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.093096 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:26.094276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.094697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.095664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:26.095983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:26.096336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.096336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:26.096336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.096336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.097488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:26.097933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:26.098406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.098406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:26.098406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.098406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:26.100533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:26.101209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:26.101453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:26.101640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:26.102417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:26.102907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:26.103164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.103942 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.104736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.105102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:26.105425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.105425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:26.106081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:26.106497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:26.107012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:26.107012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:26.107012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:30.677020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:30.677847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:30.677847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:30.678394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:30.710115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:30.710606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:30.711052 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:30.712244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:30.713194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:30.714159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:30.715194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:30.716533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:30.716533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:30.716533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:30.716533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:30.716533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:30.716533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:31.021708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:31.022169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:31.022364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:31.023496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:31.023840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:31.024087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.024930 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.025832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.026433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:31.027072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:31.027829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.028911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:31.028911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:31.028911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.028911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:31.028911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.028911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:31.071329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:31.071858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:31.072053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:31.072248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:31.072841 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:31.073015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.073596 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:31.074964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.075658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.076325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:31.076892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:31.077428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.077428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:31.077428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.077428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:31.078044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.078652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.079064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:31.090937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:31.091544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:31.092034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:31.092034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:31.092712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:31.093168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.095530 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:31.097009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.097481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.098586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:31.098974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:31.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:31.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.100594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:31.101089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:31.101654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:31.102032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:31.102268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:31.102424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.103100 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.104051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.104489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:31.104837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.104837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:31.105070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:31.105522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:31.106068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:31.106068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:31.106068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:35.677748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:35.678648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:35.678648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:35.680220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:35.716231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:35.716731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:35.717829 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:35.718843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:35.719544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:35.720250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:35.721020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:35.721955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:35.721955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:35.721955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:35.721955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:35.721955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:35.721955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:36.022456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:36.022915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:36.023047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:36.024025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:36.024486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:36.024889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.025862 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.027238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.028048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:36.028410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.028410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:36.028551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:36.029245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.029777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:36.029777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.029777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:36.072016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:36.072513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:36.072782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:36.073081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:36.073538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:36.073779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.074482 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:36.075987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.076864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.077430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:36.078072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:36.078701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.078701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:36.078701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.078701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:36.079161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.079960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.080448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:36.091373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:36.091712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:36.091954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:36.092128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:36.092446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:36.092696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.093188 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:36.094033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.094728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.095203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:36.095791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:36.096424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:36.097349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.098732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:36.101403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:36.101675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:36.101798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:36.101994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:36.102446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:36.102858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:36.103109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.103929 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.105543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.106097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:36.106537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.106537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:36.107420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:36.108203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:36.109108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:36.109108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:36.109108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:40.678383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:40.679413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:40.679531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:40.680386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:40.716926 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:40.717291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:40.718016 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:40.718713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:40.719807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:40.720409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:40.721369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:40.722429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:40.722429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:40.722429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:40.722429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:40.722429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:40.722429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:41.023464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:41.024104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:41.024322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:41.025309 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:41.025592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:41.025900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.026882 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.028390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.029203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:41.029701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.029701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:41.030100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:41.030615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.031372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:41.031372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.031372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:41.072699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:41.073115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:41.073225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:41.073385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:41.073862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:41.074187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.074710 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:41.075457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.076154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.076407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:41.076742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:41.077177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.077177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:41.077177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.077177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:41.077297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.077798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.078295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:41.092030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:41.092522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:41.092861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:41.093237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:41.094458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:41.095579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.095885 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:41.096633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.097545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:41.098124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:41.098423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.098423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:41.098423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.098423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.099014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:41.099358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:41.099731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.099731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:41.099731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.099731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:41.101904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:41.102194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:41.102301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:41.102463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:41.102855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:41.103125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:41.103313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.103745 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.104545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.104940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:41.105346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.105346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:41.105535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:41.106295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:41.106887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:41.106887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:41.106887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:45.680785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:45.680785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:45.681613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:45.681766 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:45.714422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:45.714670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:45.715528 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:45.716326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:45.716711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:45.717989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:45.718438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:45.718986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:45.718986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:45.718986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:45.718986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:45.718986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:45.718986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:46.024343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:46.024788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:46.024970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:46.025847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:46.026043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:46.026176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.026731 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.029015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.029983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:46.030895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:46.031579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.032715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:46.032715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:46.032715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.032715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:46.032715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.032715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:46.073643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:46.073970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:46.074154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:46.074353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:46.075154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:46.075493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.076400 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:46.077838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.078560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.079525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:46.080110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:46.081009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.081539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:46.092717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:46.092962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:46.093043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:46.093109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:46.093882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:46.094412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.095770 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:46.096842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.097510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.098243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:46.098612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:46.099095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.099095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:46.099095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.099095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.100075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:46.100636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:46.101208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.101208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:46.101208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.101208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:46.102447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:46.102855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:46.103044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:46.103290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:46.103995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:46.104497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:46.104766 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.105993 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.107468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:46.108209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.108209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:46.109442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:46.109963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:46.110569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:46.110569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:46.110569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:50.681236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:50.682495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:50.682750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:50.684175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:50.719813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:50.720086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:50.720983 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:50.722330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:50.722953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:50.723551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:50.723551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:50.723884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:50.725276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:50.726149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:50.726149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:50.726149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:51.025084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:51.025545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:51.025773 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:51.027234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:51.027582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:51.027882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.028914 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.030591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.031380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:51.032002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:51.032724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.033584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:51.033584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:51.033584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.033584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:51.033584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.033584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:51.074017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:51.075187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:51.075449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:51.075727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:51.076446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:51.076730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.077615 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:51.079509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.080076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.080354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:51.080837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:51.081246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.081890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.082599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:51.093242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:51.093684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:51.093834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:51.093973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:51.094536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:51.094790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.095327 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:51.096169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.096687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.097046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:51.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:51.097842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.097842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:51.097842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.097842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.098079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:51.098620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:51.099226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.099226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:51.099226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.099226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:51.102931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:51.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:51.103301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:51.103487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:51.104015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:51.104316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:51.104476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.105110 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.105953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.106350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:51.106807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.106807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:51.106901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:51.107343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:51.108034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:51.108034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:51.108034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:55.682529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:55.683422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:55.683422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:55.684020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:55.720136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:55.720663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:55.721659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:55.722788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:55.723605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:55.724221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:55.724660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:55.725288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:55.725288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:55.725288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:55.725288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:55.725288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:55.725288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:56.025539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:56.026010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:56.026177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:56.026936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:56.027145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:56.027303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.027957 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.029226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.029592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:56.029967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.029967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:56.030627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:56.031095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.031624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:56.031624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.031624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:56.075091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:56.075786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:56.076014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:56.076237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:47:56.077029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:56.077359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.078215 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:56.079941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.081064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.081494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:56.082180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:56.083022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.083740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.084666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:56.094066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:56.094489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:56.094647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:56.094894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:56.095584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:56.096016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.096974 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:47:56.098636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.099298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.100237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:56.101049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:56.101640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.101640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:56.101640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.101640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.102094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:56.102804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:56.103348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.103348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:56.103348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.103348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:47:56.103483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:47:56.103770 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:47:56.103937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:47:56.104141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:47:56.104846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:56.105221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:47:56.105442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.106221 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.107162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.107649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:47:56.108063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.108063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:47:56.108221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:47:56.108920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:47:56.109746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:47:56.109746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:47:56.109746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:00.683313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:00.684309 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:00.684309 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:00.685609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:00.718153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:00.718553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:00.719449 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:00.720511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:00.721827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:00.722479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:00.723271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:00.725434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:00.725434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:00.725434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:00.725434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:00.725434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:00.725434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:01.026379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:01.026729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:01.026893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:01.027790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:01.028121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:01.028345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.029243 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.030337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.031240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:01.031917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:01.032628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.033571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:01.033571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:01.033571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.033571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:01.033571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.033571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:01.075846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:01.076324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:01.076446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:01.076588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:01.077274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:01.077703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.078739 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:01.080038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.080961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.081569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:01.082313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:01.082962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.082962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:01.082962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.082962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:01.084142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.085057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.085622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:01.094640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:01.095309 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:01.095566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:01.095889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:01.096376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:01.096733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.097524 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:01.098898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.099554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.100232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:01.100555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:01.100907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.100907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:01.100907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.100907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.101386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:01.101862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:01.102305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.102305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:01.102305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.102305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:01.103972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:01.104278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:01.104391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:01.104549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:01.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:01.105098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:01.105276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.105723 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.106523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:01.107101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.107101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:01.107533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:01.107855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:01.108186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:01.108186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:01.108186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:05.690834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:05.691216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:05.691216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:05.691698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:05.717338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:05.717658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:05.718370 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:05.719122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:05.719505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:05.719892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:05.720505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:05.721008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:05.721008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:05.721008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:05.721008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:05.721008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:05.721008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:06.027191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:06.028017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:06.028226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:06.029791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:06.030110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:06.030407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.031291 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.033083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.033720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:06.034347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.034347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:06.035013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:06.035466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.036017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:06.036017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.036017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:06.076445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:06.077787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:06.078104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:06.078409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:06.079644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:06.080022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.081072 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:06.083132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.084175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.084901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:06.085674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:06.086198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.086198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:06.086198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.086198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:06.087306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.087816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.088397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:06.095228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:06.095936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:06.096084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:06.096267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:06.097505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:06.097979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.099139 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:06.100147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.100535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.101115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:06.101457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:06.101844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.101844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:06.101844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.101844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.103346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:06.104231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:06.105129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:06.105554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:06.105924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:06.106219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.107185 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.107882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.108540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:06.109153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:06.109942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:06.111022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:06.111022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:06.111022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:06.111022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:06.111022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:06.111022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:10.691180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:10.691939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:10.691939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:10.692748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:10.732879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:10.733237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:10.734289 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:10.736307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:10.736624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:10.736912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:10.736912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:10.738317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:10.738666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:10.739074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:10.739074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:10.739074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:11.028018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:11.028519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:11.028709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:11.029939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:11.030214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:11.030462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.031364 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.032937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.033631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:11.034317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.034317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:11.034518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:11.035290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.036155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:11.036155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.036155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:11.077727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:11.077952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:11.078105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:11.078317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:11.078971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:11.079179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.079954 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:11.081432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.082462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.082930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:11.083569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:11.084388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.084388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:11.084388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.084388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:11.084582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.085457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:11.095876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:11.096308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:11.096453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:11.096786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:11.097296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:11.097688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.098596 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:11.099769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.100161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.101178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:11.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:11.101940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.101940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:11.101940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.101940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.102700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:11.103166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:11.103635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.103635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:11.103635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.103635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:11.104775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:11.105044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:11.105158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:11.105317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:11.105714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:11.106140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:11.106363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.107071 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.107968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.108346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:11.108677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.108677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:11.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:11.109549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:11.110093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:11.110093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:11.110093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:15.691998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:15.693360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:15.693360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:15.694710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:15.730874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:15.731020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:15.731485 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:15.732277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:15.732779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:15.733188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:15.733188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:15.733470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:15.734444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:15.735469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:15.735469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:15.735469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:16.028650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:16.029200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:16.029422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:16.030383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:16.030690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:16.030929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.031652 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.033410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.034071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:16.034669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.034669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:16.035351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:16.036206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.037284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:16.037284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.037284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:16.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:16.078738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:16.078880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:16.079016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:16.079414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:16.079610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.080281 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:16.081944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.082856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.083313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:16.084049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:16.084943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.085618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.086566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:16.096348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:16.096620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:16.096735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:16.096893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:16.097301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:16.097552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.098195 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:16.099363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.099801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.100715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:16.101107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:16.101439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.101439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:16.101439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.101439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.102433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:16.102905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:16.103362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.103362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:16.103362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.103362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:16.105195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:16.105564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:16.105737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:16.106000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:16.106435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:16.106947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:16.107268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.108126 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.108984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.109332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:16.109660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.109660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:16.110040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:16.110463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:16.110932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:16.110932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:16.110932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:20.692903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:20.694180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:20.694180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:20.694817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:20.696800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:48:20.735956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:20.736427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:20.737552 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:20.738695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:20.739455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:20.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:20.740991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:20.742610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:20.742610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:20.742610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:20.742610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:20.742610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:20.742610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:21.029285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:21.029555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:21.030060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:21.031469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:21.031862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:21.032226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.033871 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.035621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.036352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:21.036972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.036972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:21.037533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:21.038516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.039152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:21.039152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.039152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:21.078976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:21.079615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:21.080040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:21.080247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:21.081193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:21.081583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.082628 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:21.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.084677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.085369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:21.085721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:21.086059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.086059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:21.086059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.086059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:21.086606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.087357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.088283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:21.096969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:21.097399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:21.097666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:21.097666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:21.098179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:21.098431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.099245 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:21.100352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.101023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.102069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:21.102517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:21.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:21.103604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.104307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:21.105654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:21.106079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:21.106079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:21.106369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:21.107034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:21.107525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:21.107664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.108477 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.109495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.109813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:21.110079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.110079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:21.110515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:21.111011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:21.111829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:21.111829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:21.111829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:25.693687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:25.694560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:25.694905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:25.696228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:25.739467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:25.739961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:25.740970 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:25.742176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:25.742724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:25.743383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:25.744019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:25.744844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:25.744844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:25.744844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:25.744844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:25.744844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:25.744844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:26.031133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:26.031559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:26.031710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:26.033212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:26.033706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:26.034120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.035444 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.038407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.038772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:26.039126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.039126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:26.040741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:26.041204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.041714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:26.041714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.041714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:26.079580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:26.079980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:26.080135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:26.080272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:26.081058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:26.081394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.082273 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:26.083218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.083855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.084278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:26.084802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:26.085366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.085366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:26.085366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.085366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:26.085842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.086827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.087532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:26.097451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:26.098023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:26.098224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:26.098444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:26.098907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:26.099319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.099901 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:26.101884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.103196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.104787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:26.105392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:26.106027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.106027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:26.106027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.106027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.108383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:26.109145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:26.109700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:26.110495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:26.111048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:26.111560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:26.111904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.112724 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.114036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.114799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:26.115700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:26.116491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:26.117309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:26.117309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:26.117309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:26.117309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:26.117309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:26.117309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:30.694474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:30.695472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:30.695472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:30.696746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:30.737282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:30.737657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:30.737934 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:30.738599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:30.739050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:30.739403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:30.739858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:30.740363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:30.740363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:30.740363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:30.740363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:30.740363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:30.740363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:31.035123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:31.035123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:31.035123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:31.036601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:31.036799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:31.036943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.037566 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.038539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.039114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:31.039645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.039645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:31.039739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:31.040435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.041220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:31.041220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.041220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:31.080149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:31.080679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:31.080869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:31.081112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:31.081665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:31.081842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.082637 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:31.083781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.084460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.085085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:31.085964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:31.086851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.086851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:31.086851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.086851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:31.087450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.088494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.089610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:31.097943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:31.098627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:31.098747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:31.098909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:31.099539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:31.100028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.100824 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:31.101625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.102602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:31.103383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.103383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:31.103383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.103383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.103669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:31.104219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:31.104717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.104717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:31.104717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.104717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:31.109461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:31.109822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:31.110197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:31.110388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:31.110860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:31.111242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:31.111483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.112184 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:31.113940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.113940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:31.115495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:31.116027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:31.116526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:31.116526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:31.116526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:35.695136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:35.696474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:35.696673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:35.697971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:35.733378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:35.733715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:35.734707 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:35.735485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:35.736161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:35.736705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:35.737534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:35.738752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:35.738752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:35.738752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:35.738752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:35.738752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:35.738752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:36.035042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:36.035736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:36.036076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:36.037249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:36.037618 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:36.037971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.039003 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.039786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.040327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:36.040903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:36.041482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.042930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:36.042930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:36.042930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.042930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:36.042930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.042930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:36.080861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:36.082450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:36.082450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:36.082450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:36.082450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:36.082450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.083020 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:36.083788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.084511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.084939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:36.085468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:36.086086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.086703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.087517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:36.098649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:36.099179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:36.099341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:36.099534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:36.100227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:36.100554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.101476 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:36.102627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.103458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.103899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:36.104621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:36.105516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:36.106388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.107317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:36.110064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:36.110547 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:36.110711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:36.110978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:36.111467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:36.111957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:36.112257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.113524 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.114099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.114492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:36.114965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.114965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:36.115113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:36.115629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:36.116182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:36.116182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:36.116182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:40.696488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:40.697546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:40.697546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:40.698593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:40.739988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:40.740442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:40.741640 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:40.743418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:40.743693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:40.743957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:40.743957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:40.745674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:40.746224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:40.746829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:40.746829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:40.746829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:41.036070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:41.036521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:41.036890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:41.037732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:41.038104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:41.038104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.038940 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.040783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.041443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:41.042303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.042303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:41.043997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:41.044943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.046142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:41.046142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.046142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:41.081520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:41.082050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:41.082213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:41.082380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:41.083050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:41.083373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.084329 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:41.085298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.085860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.086258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:41.086700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:41.087061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.087061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:41.087061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.087061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:41.087434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.088031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.088565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:41.099302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:41.099872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:41.100008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:41.100163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:41.100652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:41.100998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.101615 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:41.102510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.102941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.103716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:41.104180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:41.104521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.104521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:41.104521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.104521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.105040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:41.105520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:41.106047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.106047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:41.106047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.106047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:41.110730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:41.111121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:41.111282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:41.111426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:41.111770 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:41.112065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:41.112270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.112785 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.113651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.114619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:41.115450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:41.116468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:41.117748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:41.117748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:41.117748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:41.117748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:41.117748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:41.117748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:45.697082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:45.698150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:45.698150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:45.699320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:45.735494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:45.735887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:45.736786 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:45.737494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:45.738085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:45.738511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:45.738975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:45.739488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:45.739488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:45.739488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:45.739488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:45.739488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:45.739488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:46.036734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:46.037258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:46.037518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:46.038364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:46.038619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:46.038742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.039341 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.040751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.041209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:46.041717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.041717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:46.041938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:46.042409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.043126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:46.043126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.043126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:46.082202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:46.082559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:46.082738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:46.082918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:46.083315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:46.083519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.084160 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:46.085113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:46.086434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.087345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.087992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:46.088604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.089421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.090291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:46.100008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:46.100473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:46.100790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:46.101051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:46.101865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:46.102328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.102972 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:46.103727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.104233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.104620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:46.105007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:46.105539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.105539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:46.105539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.105539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.105936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:46.106428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:46.106984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.106984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:46.106984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.106984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:46.111194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:46.111484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:46.111484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:46.111601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:46.111959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:46.112181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:46.112334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.112819 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.113859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.114134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:46.114406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.114406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:46.115033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:46.115358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:46.115778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:46.115778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:46.115778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:50.697793 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:50.698754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:50.698754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:50.699583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:50.735326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:50.735540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:50.736139 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:50.737037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:50.737603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:50.738137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:50.739349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:50.740655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:50.740655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:50.740655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:50.740655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:50.740655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:50.740655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:51.037516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:51.037915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:51.038025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:51.038924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:51.039159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:51.039334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.039982 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.040999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.041284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:51.041518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.041518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:51.042357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:51.042926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.043894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:51.043894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.043894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:51.082801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:51.083233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:51.083440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:51.083658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:51.084741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:51.085292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.086537 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:51.087424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.087858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.088273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:51.088611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:51.088986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.088986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:51.088986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.088986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:51.089327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.089798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.090290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:51.100533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:51.101010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:51.101214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:51.101433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:51.101971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:51.102223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.102872 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:51.104221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.105045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.105712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:51.106299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:51.106976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.106976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:51.106976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.106976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:51.108172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:51.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:51.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:51.111890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:51.111890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:51.111890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:51.112101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:51.112523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:51.112923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:51.113209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.114136 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.115482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.115878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:51.116237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.116237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:51.117222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:51.117694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:51.118279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:51.118279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:51.118279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:55.698832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:55.699671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:55.699671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:55.700556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:55.738892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:55.739393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:55.740360 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:55.741087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:55.741984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:55.742727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:55.743567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:55.744517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:55.744517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:55.744517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:55.744517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:55.744517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:55.744517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:56.038338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:56.038705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:56.038813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:56.039993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:56.040209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:56.040625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.041364 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.042373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:56.043285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.043285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:56.043506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:56.044091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.044602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:56.044602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.044602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:56.083429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:56.084380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:56.084667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:56.084942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:48:56.085641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:56.086087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.086915 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:56.087903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.088419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.088954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:56.089880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:56.090682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.091560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.092651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:56.101248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:56.101784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:56.101947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:56.102050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:56.102641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:56.103121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.104038 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:48:56.105257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.105814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.106310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:56.106856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:56.107302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.107302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:56.107302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.107302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.107717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:56.108553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:56.109209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.109209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:56.109209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.109209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:48:56.111922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:48:56.112401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:48:56.112532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:48:56.112777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:48:56.113289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:56.113654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:48:56.113928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.114705 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.116334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.116901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:48:56.117655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.117655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:48:56.117922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:48:56.118468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:48:56.119075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:48:56.119075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:48:56.119075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:00.699134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:00.700179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:00.700179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:00.701250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:00.747063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:00.747738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:00.748118 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:00.749091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:00.749887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:00.750900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:00.751904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:00.753204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:00.753204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:00.753204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:00.753204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:00.753204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:00.753204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:01.038815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:01.039349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:01.039557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:01.051973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:01.052205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:01.052517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.054000 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.014 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.055642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.056011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:01.056329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.056329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:01.057456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:01.057930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.058425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:01.058425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.058425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:01.084222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:01.084718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:01.084900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:01.085043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:01.085724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:01.085969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.086840 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:01.088452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.089020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.089540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:01.089945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:01.090284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.090284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:01.090284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.090284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:01.090508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.090970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.091443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:01.101667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:01.103110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:01.103110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:01.103110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:01.103601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:01.104010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.105196 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:01.106245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.106897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.107529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:01.108119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:01.108796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.108796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:01.108796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.108796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.109189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:01.109712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:01.110310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.110310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:01.110310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.110310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:01.112604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:01.112969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:01.113132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:01.113480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:01.114125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:01.114392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:01.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.115158 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.116147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.116624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:01.117189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.117189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:01.117573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:01.118100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:01.118594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:01.118594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:01.118594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:05.699708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:05.700644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:05.700644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:05.702240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:05.738311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:05.738455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:05.739158 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:05.740048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:05.740464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:05.740734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:05.740734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:05.741007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:05.741353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:05.741749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:05.741749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:05.741749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:06.039569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:06.040151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:06.040356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:06.041393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:06.041905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:06.042343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.043492 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.045459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.046471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:06.047114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.047114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:06.047253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:06.047868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.048355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:06.048355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.048355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:06.084823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:06.085295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:06.085559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:06.085870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:06.096532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:06.096532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.097241 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:06.098951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.099320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:06.100521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:06.100808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.100808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:06.100808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.100808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:06.101213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.101528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.102087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:06.102621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:06.103008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:06.103416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:06.103416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:06.103810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:06.104067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.104671 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:06.105528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.106377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.106846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:06.107396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:06.108143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:06.108851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.109967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:06.113092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:06.113437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:06.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:06.113733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:06.114313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:06.114599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:06.114868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.115639 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.117846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:06.118177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.118177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:06.119079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:06.119704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:06.120285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:06.120285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:06.120285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:10.700485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:10.701459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:10.701459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:10.702688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:10.737117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:10.737919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:10.739239 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:10.741024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:10.741638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:10.742273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:10.742273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:10.743551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:10.744691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:10.745708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:10.745708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:10.745708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:11.040212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:11.041036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:11.041271 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:11.042338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:11.042597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:11.042784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.043491 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.046097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.046794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:11.047405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.047405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:11.048740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:11.049618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.050558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:11.050558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.050558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:11.085541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:11.086092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:11.086275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:11.086500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:11.087194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:11.087552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.088645 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:11.089984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.090749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.091335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:11.092135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:11.092720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.092720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:11.092720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.092720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:11.092974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.093627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.094158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:11.103107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:11.103383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:11.103465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:11.103574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:11.104023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:11.104318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.105131 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:11.105743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:11.106402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:11.107042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:11.107611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:11.108180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.109193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:11.113600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:11.137472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:11.137472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:11.137472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:11.137472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:11.137472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:11.137472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:11.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:11.138310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.138691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:11.139133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:11.139698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:15.701185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:15.701850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:15.701850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:15.702751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:15.735740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:15.736267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:15.737476 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:15.738286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:15.738861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:15.739523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:15.740164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:15.740921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:15.740921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:15.740921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:15.740921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:15.740921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:15.740921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:16.041042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:16.041473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:16.041714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:16.042946 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:16.043297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:16.043543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.044446 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.046256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.046722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:16.047077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.047077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:16.047425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:16.047865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.048327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:16.048327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.048327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:16.086158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:16.086685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:16.086792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:16.086966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:16.087462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:16.087827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.088586 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:16.089953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.090495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.091203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:16.092414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:16.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.094278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:16.103443 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:16.103731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:16.103922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:16.104129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:16.104613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:16.105034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.105940 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:16.107323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.107737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.109080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:16.109430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:16.109904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.109904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:16.109904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.109904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.111382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:16.111864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:16.112379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.112379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:16.112379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.112379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:16.133917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:16.134210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:16.134365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:16.134611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:16.135260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:16.135987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:16.136104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.137454 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.138880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.139305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:16.139843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.139843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:16.140429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:16.140930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:16.141573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:16.141573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:16.141573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:20.701882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:20.703166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:20.703166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:20.703329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:20.709259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=241 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:49:20.739161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:20.739483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:20.740258 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:20.741298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:20.742002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:20.742618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:20.743404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:20.744335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:20.744335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:20.744335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:20.744335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:20.744335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:20.744335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:21.041839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:21.042193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:21.042419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:21.043553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:21.043943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:21.044495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.045517 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.047169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.048110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:21.048887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.048887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:21.049067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:21.049956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.050897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:21.050897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.050897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:21.086879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:21.087323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:21.087520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:21.087730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:21.088510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:21.088922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.089969 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:21.091345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.091893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.093093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:21.093512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:21.094028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.094028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:21.094028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.094028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:21.094486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.095009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.095497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:21.103898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:21.104159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:21.104308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:21.104461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:21.105305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:21.105775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.106572 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:21.107397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:21.108309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:21.109208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:21.110066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:21.111045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.112139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:21.134409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:21.135111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:21.135339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:21.135611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:21.136244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:21.136545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:21.136712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.137544 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.139290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.139848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:21.140327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.140327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:21.162173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:21.162896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:21.164028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:21.164028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:21.164028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:25.702826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:25.703863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:25.703863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:25.705446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:25.739888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:25.740318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:25.741168 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:25.741968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:25.742661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:25.743373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:25.744235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:25.745296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:25.745296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:25.745296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:25.745296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:25.745296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:25.745296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:26.042431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:26.043116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:26.043221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:26.044313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:26.044743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:26.045103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.045942 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.047602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.048295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:26.048888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.048888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:26.050293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:26.051068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.051991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:26.051991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.051991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:26.087439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:26.087940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:26.088048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:26.088254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:26.089072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:26.089448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.091676 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:26.092693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:26.093618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.094648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.095418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:26.096126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.096900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.097469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:26.104343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:26.105775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:26.105910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:26.106019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:26.106652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:26.107079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.107870 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:26.108848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.109544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.110041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:26.110600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:26.111505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:26.112311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.113252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:26.135280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:26.135951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:26.136134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:26.136390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:26.137081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:26.137292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:26.137359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.137965 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.139010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.139965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:26.140637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:26.141376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:26.142357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:26.142357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:26.142357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:26.142357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:26.142357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:26.142357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:30.703630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:30.704230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:30.704339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:30.705310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:30.739897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:30.740332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:30.741492 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:30.742646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:30.743222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:30.744103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:30.744717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:30.745509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:30.745509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:30.745509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:30.745509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:30.745509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:30.745509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:31.043207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:31.043887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:31.044125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:31.045919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:31.045919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:31.045919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.046259 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.047918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.048337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:31.048692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.048692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:31.050163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:31.050676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.051378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:31.051378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.051378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:31.088136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:31.088583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:31.088781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:31.088996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:31.089951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:31.090054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.091077 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:31.092475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.093247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.093776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:31.094233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:31.094730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.094730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:31.094730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.094730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:31.095299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.095704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.096194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:31.104932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:31.105314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:31.105476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:31.105663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:31.106362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:31.106871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.107651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:31.108857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.109588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.110848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:31.111246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:31.111613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.111613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:31.111613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.111613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.112054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:31.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:31.113109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.113109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:31.113109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.113109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:31.135957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:31.136386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:31.136544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:31.136789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:31.137379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:31.137616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:31.137756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.138381 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.139910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.140516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:31.141164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.141164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:31.142002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:31.142491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:31.143037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:31.143037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:31.143037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:35.704367 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:35.705269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:35.705355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:35.706635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:35.742125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:35.742125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:35.742357 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:35.743473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:35.744133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:35.744795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:35.745566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:35.746476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:35.746476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:35.746476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:35.746476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:35.746476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:35.746476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:36.044065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:36.044747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:36.045099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:36.046122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:36.046409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:36.046707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.047780 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.049051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.049976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:36.050702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:36.051518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.052443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:36.052443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:36.052443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.052443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:36.052443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.052443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:36.088805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:36.089432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:36.089652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:36.089915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:36.090328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:36.090559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.091182 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:36.092480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.093500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.094021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:36.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:36.095619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.096395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.097320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:36.105342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:36.105579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:36.105695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:36.105852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:36.106332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:36.106692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.107834 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:36.108730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.109447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.110227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:36.110924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:36.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:36.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.112087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:36.113077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:36.114236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.114236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:36.114236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.114236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:36.136425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:36.136913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:36.137163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:36.137420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:36.137974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:36.138303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:36.138406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.138945 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.140867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:36.141474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.141474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:36.141890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:36.142658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:36.143557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:36.143557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:36.143557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:40.705150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:40.706361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:40.706361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:40.707569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:40.744227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:40.744511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:40.745430 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:40.746521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:40.747244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:40.748039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:40.748873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:40.749862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:40.749862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:40.749862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:40.749862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:40.749862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:40.749862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:41.044936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:41.045323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:41.045601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:41.046892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:41.047228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:41.047482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.048486 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.050252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.051135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:41.051975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.051975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:41.052561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:41.053099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.053606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:41.053606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.053606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:41.089406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:41.089842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:41.090043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:41.090217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:41.091002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:41.091305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.092176 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:41.093646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.094975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.095549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:41.096447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:41.097047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.097047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:41.097047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.097047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:41.097753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.098502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.099199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:41.105856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:41.106104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:41.106104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:41.106185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:41.106641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:41.107028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.107795 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:41.109180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.109611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.110714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:41.111084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:41.111436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.111436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:41.111436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.111436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.112723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:41.113495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:41.114111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.114111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:41.114111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.114111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:41.137086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:41.137597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:41.137897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:41.138237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:41.139379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:41.140264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:41.140549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.141674 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.143414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.144472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:41.145028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.145028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:41.145419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:41.146106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:41.146823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:41.146823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:41.146823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:45.705779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:45.706384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:45.706510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:45.707398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:45.741522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:45.741824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:45.742586 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:45.743664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:45.744305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:45.744937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:45.745615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:45.746579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:45.746579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:45.746579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:45.746579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:45.746579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:45.746579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:46.045449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:46.046114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:46.046329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:46.047432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:46.047797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:46.048039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.048977 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.050483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.051028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:46.051565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.051565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:46.052237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:46.052793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.053263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:46.053263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.053263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:46.089925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:46.090384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:46.090556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:46.090737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:46.091214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:46.091442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.092119 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:46.093309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.093910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.094618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:46.095135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:46.095647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.095647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:46.095647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.095647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:46.095807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.096287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.096839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:46.106440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:46.106899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:46.107122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:46.107313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:46.107874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:46.108239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.109017 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:46.110189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.110675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.111599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:46.111948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:46.112397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.112397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:46.112397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.112397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.113350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:46.114050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:46.114917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.114917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:46.114917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.114917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:46.137605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:46.138110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:46.138369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:46.138593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:46.139304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:46.139800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:46.140004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.140864 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.142149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.142682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:46.144292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:46.145694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:46.145694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:46.145694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:46.145694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:46.145694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:46.145694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:50.706500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:50.707163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:50.707410 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:50.709069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:50.743174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:50.743521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:50.744288 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:50.744998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:50.745546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:50.746128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:50.746824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:50.747600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:50.747600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:50.747600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:50.747600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:50.747600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:50.747600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:51.046274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:51.046959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:51.047155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:51.048313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:51.048677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:51.049077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.050261 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.052096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.052710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:51.053396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.053396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:51.053875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:51.054638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.055660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:51.055660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.055660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:51.090608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:51.091179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:51.091304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:51.091398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:51.092130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:51.092469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.093563 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:51.096238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:51.097523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.098408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.099148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:51.099810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.100670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.101641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:51.106948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:51.107303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:51.107441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:51.107599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:51.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:51.108365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:51.109912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.110333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.110995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:51.111333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:51.111717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.111717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:51.111717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.111717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:51.112773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:51.113331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.113331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:51.113331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.113331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:51.138299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:51.138575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:51.138667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:51.138845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:51.139570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:51.140004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:51.140233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.140989 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.141715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.142251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:51.142631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:51.143112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:51.143653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:51.143653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:51.143653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:51.143653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:51.143653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:51.143653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:55.707839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:55.708955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:55.708955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:55.710269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:55.747247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:55.747770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:55.749034 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:55.750658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:55.751356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:55.752082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:55.752082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:55.752209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:55.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:55.753387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:55.753387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:55.753387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:56.047069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:56.047992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:56.048246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:56.049676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:56.050202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:56.050479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.051610 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.053358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.054073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:56.054665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.054665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:56.055059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:56.055639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.056197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:56.056197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.056197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:56.091411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:56.092160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:56.092356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:56.092564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:49:56.093377 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:56.093717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.094872 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:56.096394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.097585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.098235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:56.099164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:56.100196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.101151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.102484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:56.107423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:56.107796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:56.107894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:56.108043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:56.108619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:56.109046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.110070 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:49:56.110894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.111183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.112516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:56.112782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:56.113297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.113297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:56.113297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.113297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.114370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:56.114897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:56.115581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.115581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:56.115581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.115581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:49:56.139124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:49:56.139999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:49:56.140148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:49:56.140493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:49:56.140960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:56.141212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:49:56.141420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.142140 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.143825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.144343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:49:56.144826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:49:56.145401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:49:56.146021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:49:56.146021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:49:56.146021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:49:56.146021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:49:56.146021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:49:56.146021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:00.708542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:00.709516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:00.709516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:00.710167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:00.741322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:00.741733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:00.744480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:00.745109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:00.745420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:00.745742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:00.746246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:01.047712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:01.048249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:01.048449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:01.049513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:01.049845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:01.050015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.050752 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.052317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.053311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:01.054046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:01.055165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.056442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:01.056442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:01.056442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.056442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:01.056442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.056442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:01.092117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:01.092654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:01.092797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:01.092916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:01.094020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:01.094580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.095645 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:01.096623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:01.098468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.099543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:01.102381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.103571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.104073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:01.107861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:01.108126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:01.108243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:01.108388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:01.109352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.110465 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:01.111535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.111972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.113104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:01.113478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:01.113875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.113875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:01.113875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.113875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.115181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:01.115645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:01.116148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.116148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:01.116148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.116148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:01.139899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:01.140234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:01.140976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:01.141625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:01.142660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:01.142660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.143638 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.145946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:01.146555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.146555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:01.147267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:01.148228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:01.149093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:01.149093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:01.149093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:05.709176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:05.710317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:05.710317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:05.711603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:05.752194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:05.752689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:05.753649 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:05.754727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:05.755342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:05.755723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:05.755723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:05.755831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:05.756376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:05.757040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:05.757040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:05.757040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:06.048569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:06.049175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:06.049371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:06.050581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:06.050993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:06.051235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.052121 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.053670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.054242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:06.054523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.054523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:06.055437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:06.055832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.056418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:06.056418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.056418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:06.092942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:06.093508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:06.093687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:06.093938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:06.094814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:06.095201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.096164 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:06.097497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.098313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.099128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:06.100454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:06.101655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.102088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.102587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:06.108213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:06.108604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:06.108702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:06.108850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:06.109509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:06.109873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.110619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:06.111411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.111851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.112335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:06.112651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:06.113058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.113058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:06.113058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.113058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.113511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:06.114562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:06.115083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.115083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:06.115083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.115083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:06.140244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:06.140581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:06.140735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:06.141069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:06.141641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:06.141914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:06.142035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.142521 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.143829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:06.145560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:06.146258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:06.147071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:06.147071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:06.147071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:06.147071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:06.147071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:06.147071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:10.710180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:10.711457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:10.711457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:10.711795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:10.745385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:10.745814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:10.746660 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:10.747552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:10.748089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:10.748486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:10.748956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:10.749579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:10.749579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:10.749579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:10.749579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:10.749579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:10.749579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:11.049298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:11.049880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:11.050102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:11.051251 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:11.051748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:11.052143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.053222 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.054924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.056079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:11.056603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.056603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:11.056995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.059575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:11.059575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.059575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:11.093541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:11.093959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:11.094184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:11.094184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:11.094786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:11.095138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.095992 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:11.097335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.098146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.098988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:11.100318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:11.101167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.101938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.102413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:11.108769 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:11.109164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:11.109287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:11.109450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:11.109964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:11.110353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.111123 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:11.113069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.113446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.114775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:11.115115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:11.115497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.115497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:11.115497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.115497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.116777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:11.117213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:11.117670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.117670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:11.117670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.117670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:11.140861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:11.141343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:11.141620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:11.141951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:11.142628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:11.143119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:11.143395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.144401 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.145842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.147173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:11.148118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:11.149277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:11.151020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:11.151020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:11.151020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:11.151020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:11.151020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:11.151020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:15.710987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:15.711892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:15.711892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:15.713253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:15.754532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:15.754787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:15.755560 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:15.756712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:15.757242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:15.758025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:15.758488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:15.758941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:15.758941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:15.758941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:15.758941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:15.758941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:15.758941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:16.049784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:16.050382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:16.050605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:16.051647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:16.052029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:16.052301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.053234 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.054781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.055391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:16.056029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.056029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:16.056584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:16.057359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.058171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:16.058171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.058171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:16.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:16.094853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:16.094988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:16.095125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:16.095684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:16.096069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.097115 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:16.098135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.098888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.099347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:16.099939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:16.100704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.101450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:16.109248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:16.109696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:16.109886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:16.110212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:16.110572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:16.110854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.111469 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:16.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.113199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.114346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:16.114720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:16.115226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.115226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:16.115226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.115226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.116314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:16.116807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:16.117355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.117355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:16.117355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.117355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:16.141539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:16.142236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:16.142493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:16.142863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:16.143543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:16.144146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:16.144423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.147062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.147860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:16.148533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.148533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:16.149273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:16.150857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:16.151692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:16.151692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:16.151692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:20.711562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:20.712488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:20.712488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:20.713222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:20.714617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=239 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:50:20.749139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:20.749412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:20.750076 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:20.750885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:20.751411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:20.751963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:20.752793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:20.754328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:20.754328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:20.754328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:20.754328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:20.754328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:20.754328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:21.050560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:21.051155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:21.051434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:21.052535 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:21.052887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:21.053064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.053845 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.055339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.055790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:21.056187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.056187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:21.057215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:21.058679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.059512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:21.059512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.059512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:21.095315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:21.095848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:21.095984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:21.096108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:21.096902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:21.097339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.098414 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:21.099265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.099673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.100206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:21.100572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:21.101097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.101097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:21.101097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.101097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:21.101336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.102137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.102867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:21.109612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:21.109889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:21.109998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:21.110130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:21.110534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:21.110886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.111519 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:21.112713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.113447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:21.114665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:21.115301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.115301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:21.115301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.115301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.115963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:21.117602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:21.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:21.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:21.142124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:21.142556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:21.142842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:21.143120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:21.143748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:21.144266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:21.144596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.145670 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.147346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.148224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:21.148860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.148860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:21.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:21.151049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:21.152027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:21.152027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:21.152027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:25.712275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:25.713170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:25.713170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:25.714306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:25.939530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:25.939984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:25.940843 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.228 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:25.941805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:25.942453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:25.943729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:25.945454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:25.947489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:25.947489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:25.947489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:25.947489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:25.947489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:25.947489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:26.051314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:26.052081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:26.052313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:26.053400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:26.053640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:26.053810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.054504 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.056223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.056870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:26.057463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.057463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:26.057996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:26.058448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.059015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:26.059015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.059015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:26.096108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:26.096728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:26.096968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:26.097168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:26.097723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:26.097971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.098619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:26.100034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.100932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.101677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:26.102743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:26.103950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.103950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:26.103950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.103950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:26.105536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.106929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:26.110027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:26.110256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:26.110318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:26.110425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:26.110784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:26.111047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.111503 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:26.112069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.112510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.112792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:26.113125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:26.113504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:26.113934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.114455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:26.142539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:26.143099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:26.143292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:26.143565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:26.144090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:26.144462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:26.144694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.145478 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.146936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.147596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:26.148309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.148309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:26.148593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:26.149848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:26.151093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:26.151093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:26.151093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:30.298132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:30.298981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:30.299113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:30.308901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 01:50:30.309038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 01:50:30.309252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:50:30.343561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.344156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.345013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.345464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:50:30.347877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.349678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.350424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:50:30.378966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.378966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.379288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.379288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.379970 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.380906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.381484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.382177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.382177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.382177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.382177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:50:30.419416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.419836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.420378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.420574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.421425 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.422434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.423003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.423622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.423622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.423622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.423622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:50:30.447431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.447641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.448178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.448377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.449134 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.450065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.450570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.451355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.451355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.451355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.451355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:50:30.528613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.529146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.529663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.530094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.530420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.531739 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.532000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 01:50:30.533637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.534199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.534833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:50:30.644392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.644861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.645389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.645876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.647030 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.648152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.648694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.649346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.649346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.649346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.649346 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.349 seconds
May 05 01:50:30.712985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:30.713673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:30.713939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:30.715462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:30.755784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:30.756240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.757409 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.758505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.759155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:30.759748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:30.760728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:30.761815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:30.761815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:30.761815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:30.761815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:30.761815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:30.761815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:31.053835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:31.053835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:31.053835 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:31.054708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:31.055134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:31.055385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.056458 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.058248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.058620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:31.058981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.058981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:31.059420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:31.060315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.060816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:31.060816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.060816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:31.097104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:31.097947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:31.098170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:31.098387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:31.099295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:31.099533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.100377 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:31.102242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.103015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.103955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:31.104822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:31.105504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.106343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.107669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:31.110789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:31.111468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:31.111639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:31.111884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:31.112574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:31.113041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.113931 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:31.114903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.115449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.116107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:31.116561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:31.117095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.117095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:31.117095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.117095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.117554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:31.118196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:31.118910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.118910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:31.118910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.118910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:31.143000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:31.143959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:31.143959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:31.143959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:31.144329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:31.144715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:31.145020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.145854 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.146881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.147530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:31.148155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:31.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:31.150329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:31.150329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:31.150329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:31.150329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:31.150329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:31.150329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:35.713604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:35.714463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:35.714543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:35.715851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:35.752196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:35.752630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:35.753791 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:35.754458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:35.755118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:35.755846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:35.756593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:35.757518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:35.757518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:35.757518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:35.757518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:35.757518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:35.757518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:36.052962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:36.053397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:36.053632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:36.054635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:36.054925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:36.055143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.056035 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.057862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.058261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:36.058735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.058735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:36.060043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:36.060583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.061267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:36.061267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.061267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:36.097882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:36.098330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:36.098522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:36.098718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:36.099416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:36.100062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.100870 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:36.102218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.102603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.103775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:36.104136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:36.104459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.104459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:36.104459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.104459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:36.105039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.105438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.105898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:36.111497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:36.111800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:36.111916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:36.112045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:36.112364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:36.112592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.113078 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:36.114921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.115487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.116259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:36.116966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:36.117636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.117636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:36.117636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.117636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.118155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:36.120502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:36.123237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.123237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:36.123237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.123237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:36.143449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:36.143881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:36.144123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:36.144920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:36.144920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:36.144920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:36.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.147188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:36.148595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:36.149436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:36.150273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:36.150273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:36.150273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:36.150273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:36.150273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:36.150273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:40.714325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:40.715365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:40.715365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:40.716475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:40.752567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:40.752989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:40.753858 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:40.754633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:40.755050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:40.755431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:40.755925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:40.756464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:40.756464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:40.756464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:40.756464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:40.756464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:40.756464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:41.053564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:41.054049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:41.054262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:41.055297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:41.055675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:41.056059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.057252 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.058706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.060073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:41.061648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.061648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:41.061940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:41.062654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.064839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:41.064839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.064839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:41.098439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:41.099998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:41.099998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:41.099998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:41.100583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:41.101048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.104806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.106224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.107895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:41.108696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:41.109489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.109489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:41.109489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.109489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:41.110462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.110827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.111196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:41.111921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:41.112386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:41.112386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:41.112386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:41.112873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:41.113258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.114001 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:41.114542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:41.115275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:41.115936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:41.116596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:41.117182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.118245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:41.143917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:41.144407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:41.144642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:41.144998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:41.145473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:41.145869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:41.146093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.146806 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.147613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.147989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:41.148343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.148343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:41.148627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:41.149066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:41.149538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:41.149538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:41.149538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:45.715177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:45.716911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:45.716911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:45.717827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:45.755595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:45.756161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:45.757217 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:45.758132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:45.758749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:45.759398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:45.760161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:45.760947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:45.760947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:45.760947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:45.760947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:45.760947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:45.760947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:46.054387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:46.054812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:46.054948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:46.056004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:46.056349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:46.056608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.057545 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.058638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.059244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:46.059595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.059595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:46.059814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:46.060810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.061677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:46.061677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.061677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:46.099549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:46.100229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:46.100444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:46.100665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:46.101411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:46.101782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.102887 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:46.104416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.105354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.105888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:46.106552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:46.107380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.107380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:46.107380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.107380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:46.108161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.109321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.110255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:46.112307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:46.112682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:46.112817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:46.112950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:46.113322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:46.113558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.114105 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:46.114931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.115371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.116039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:46.116403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:46.116792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.116792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:46.116792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.116792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.117108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:46.117618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:46.118229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.118229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:46.118229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.118229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:46.144477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:46.145007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:46.145194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:46.145445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:46.146086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:46.146461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:46.146729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.147602 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.148972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.149674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:46.150358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.150358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:46.150504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:46.151365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:46.152384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:46.152384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:46.152384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:50.716173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:50.717078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:50.717078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:50.718037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:50.748995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:50.749399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:50.750353 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:50.751171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:50.751827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:50.752457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:50.753313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:50.754271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:50.754271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:50.754271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:50.754271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:50.754271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:50.754271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:51.055161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:51.055597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:51.055679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:51.056595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:51.056859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:51.057019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.057586 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.058589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.059287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:51.059908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:51.060680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.061587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:51.061587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:51.061587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.061587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:51.061587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.061587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:51.100007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:51.100502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:51.100714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:51.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:51.101893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:51.102282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.103273 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:51.104679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.105547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.106175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:51.106840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:51.107479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.107479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:51.107479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.107479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:51.108000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.108737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.109508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:51.112864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:51.113189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:51.113347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:51.113455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:51.113839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:51.114088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:51.115357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.115790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.116361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:51.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:51.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.117458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:51.117958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:51.118472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.118472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:51.118472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.118472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:51.145372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:51.146056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:51.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:51.146504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:51.147311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:51.147809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:51.147979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.148609 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.149992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.150634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:51.151286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.151286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:51.152070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:51.152866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:51.153695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:51.153695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:51.153695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:55.716937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:55.717587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:55.717681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:55.719095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:55.753536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:55.753942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:55.754904 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:55.755648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:55.756066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:55.756633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:55.757103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:55.757661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:55.757661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:55.757661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:55.757661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:55.757661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:55.757661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:56.055918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:56.056299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:56.056412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:56.057453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:56.057785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:56.058073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.058863 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.059955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.060519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:56.061157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.061157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:56.062015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:56.062826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.063939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:56.063939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.063939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:56.100608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:56.101509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:56.101871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:56.102073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:50:56.102750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:56.103013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.103700 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:56.105605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.106406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:56.107471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:56.108083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.109659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:56.113444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:56.113828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:56.113950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:56.114081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:56.114627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:56.114890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.115442 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:50:56.116181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.116579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.117185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:56.117572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:56.117974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.117974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:56.117974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.117974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:56.118718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:56.119246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.119246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:56.119246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.119246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:50:56.146035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:50:56.146438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:50:56.146659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:50:56.146975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:50:56.147504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:56.147715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:50:56.147843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.148420 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.149952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.150537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:50:56.151124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.151124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:50:56.151562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:50:56.152209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:50:56.152777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:50:56.152777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:50:56.152777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:00.717532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:00.718860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:00.718860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:00.720094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:00.760371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:00.760734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:00.761306 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:00.762202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:00.762987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:00.764293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:00.764944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:00.765853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:00.765853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:00.765853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:00.765853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:00.765853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:00.765853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:01.056541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:01.057133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:01.057277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:01.058279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:01.058508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:01.058661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.059568 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.061025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.061588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:01.062132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.062132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:01.062923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:01.063671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.064389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:01.064389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.064389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:01.101258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:01.101731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:01.101963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:01.102102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:01.102732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:01.102974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.103746 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:01.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.105877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.107052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:01.107507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:01.107879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.107879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:01.107879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.107879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:01.108721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.109421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.110087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:01.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:01.114396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:01.114523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:01.114621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:01.115279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:01.115841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.116664 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:01.117689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.118165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.119849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:01.120296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:01.120724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.120724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:01.120724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.120724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.122034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:01.122724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:01.123615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.123615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:01.123615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.123615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:01.146625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:01.147193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:01.147432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:01.147710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:01.148560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:01.149126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:01.149373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.150228 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.151927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.152416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:01.153335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.153335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:01.153975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:01.154843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:01.156002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:01.156002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:01.156002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:05.718200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:05.718841 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:05.718841 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:05.719608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:05.749101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:05.749101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:05.749859 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:05.750717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:05.751740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:05.752607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:05.753439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:05.754501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:05.754501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:05.754501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:05.754501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:05.754501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:05.754501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:06.057269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:06.057721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:06.057985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:06.059289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:06.059784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:06.060581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.061178 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.062592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.063235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:06.063808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.063808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:06.064625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:06.065442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.066664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:06.066664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.066664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:06.101903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:06.102490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:06.102689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:06.102951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:06.103581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:06.104079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.104574 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:06.105428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:06.106631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.107644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.108550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:06.109293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.110174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.110735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:06.114501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:06.114962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:06.115173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:06.115387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:06.115795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:06.116096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.116743 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:06.118589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.119335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.120069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:06.120695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:06.121454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.121454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:06.121454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.121454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.122037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:06.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:06.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:06.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:06.147190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:06.147638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:06.147823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:06.148055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:06.148585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:06.148867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:06.149021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.149587 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.150627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:06.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:06.151649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:06.151996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:06.152372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:06.152372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:06.152372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:10.719037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:10.719877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:10.719877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:10.720455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:10.753698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:10.754125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:10.754960 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:10.756409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:10.757475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:10.758306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:10.759168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:10.760185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:10.760185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:10.760185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:10.760185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:10.760185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:10.760185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:11.057915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:11.058392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:11.058596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:11.059860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:11.060203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:11.060432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.061281 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.063321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.063994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:11.064529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.064529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:11.064978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:11.065692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.066560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:11.066560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.066560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:11.102419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:11.103025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:11.103233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:11.103431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:11.104154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:11.104519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.107065 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:11.108360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.109283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.109800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:11.110427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:11.111416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.112224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.113561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:11.115059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:11.115325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:11.115399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:11.115503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:11.115960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:11.116274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.116843 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:11.117379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.117917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.118181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:11.118505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:11.118880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:11.119243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.119857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:11.147732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:11.148223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:11.148413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:11.148695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:11.149412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:11.149845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:11.150175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.151081 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.152285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.152661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:11.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:11.153817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:11.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:11.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:11.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:11.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:11.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:11.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:15.720000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:15.721111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:15.721111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:15.722167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:15.755200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:15.755481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:15.756313 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:15.757004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:15.757713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:15.758382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:15.759213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:15.760167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:15.760167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:15.760167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:15.760167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:15.760167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:15.760167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:16.058731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:16.059318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:16.059443 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:16.060566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:16.060985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:16.061242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.062174 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.064111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.064722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:16.065377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.065377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:16.066748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:16.067623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.068510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:16.068510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.068510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:16.103192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:16.103625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:16.103735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:16.103919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:16.104540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:16.104884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.105617 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:16.106446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.107241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.107961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:16.108556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:16.109199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.109854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.110750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:16.115554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:16.115913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:16.116026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:16.116152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:16.116562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:16.116745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.117263 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:16.118009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.118409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.118890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:16.119239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:16.119632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.119632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:16.119632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.119632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.119971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:16.120598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:16.121154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.121154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:16.121154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.121154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:16.148396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:16.148936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:16.149062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:16.149315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:16.149962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:16.150374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:16.150612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.151514 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.152323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.153063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:16.153678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:16.154410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:16.155504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:16.155504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:16.155504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:16.155504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:16.155504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:16.155504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:20.720904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:20.721842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:20.721842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:20.722878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:20.731074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=435 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:51:20.763866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:20.764253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:20.765163 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:20.766342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:20.767159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:20.767894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:20.768709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:20.769941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:20.769941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:20.769941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:20.769941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:20.769941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:20.769941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:21.059543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:21.060107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:21.060341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:21.061787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:21.062269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:21.062630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.063712 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.065352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.065745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:21.066091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.066091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:21.067347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:21.068237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.069176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:21.069176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.069176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:21.103842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:21.104324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:21.104567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:21.104821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:21.106473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:21.106473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.107236 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:21.108238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.109382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.110052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:21.110692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:21.111390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.111390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:21.111390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.111390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:21.113487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.114561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.115801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:21.116201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:21.116806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:21.116882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:21.117010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:21.117275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:21.117482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.118003 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:21.118684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.119004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.119493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:21.119801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:21.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:21.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.120498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:21.120922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:21.121300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.121300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:21.121300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.121300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:21.149032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:21.149497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:21.149779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:21.150132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:21.150804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:21.151275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:21.151560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.152578 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.154310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.155061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:21.155891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.155891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:21.156577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:21.157462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:21.158467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:21.158467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:21.158467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:25.721523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:25.722178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:25.722275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:25.723510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:25.755703 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:25.756270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:25.757148 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:25.757902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:25.758203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:25.758483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:25.758483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:25.758656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:25.759064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:25.759494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:25.759494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:25.759494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:26.060402 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:26.060913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:26.061117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:26.062268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:26.062604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:26.062884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.063778 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.065579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.066216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:26.066773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.066773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:26.067351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:26.068184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.069069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:26.069069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.069069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:26.104604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:26.104945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:26.105142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:26.105341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:26.105989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:26.106280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.107113 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:26.108228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.109117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.109579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:26.110230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:26.111071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.111714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.112622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:26.116615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:26.116937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:26.117119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:26.117310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:26.117914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:26.118317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.119047 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:26.119738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:26.120658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:26.121284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.121632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.121970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:26.122443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:26.122998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.122998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:26.122998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.122998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:26.149555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:26.149969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:26.150111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:26.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:26.151023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:26.151424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:26.151643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.152509 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:26.155505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:26.156175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:26.157174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:26.157174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:26.157174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:26.157174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:26.157174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:26.157174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:30.722248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:30.723260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:30.723260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:30.724748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:30.758102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:30.758286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:30.758877 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:30.759539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:30.760157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:30.760701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:30.761436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:30.762234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:30.762234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:30.762234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:30.762234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:30.762234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:30.762234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:31.061483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:31.061994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:31.062155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:31.063650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:31.064154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:31.064729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.065875 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.067163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.068128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:31.069058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:31.070037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.071155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:31.071155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:31.071155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.071155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:31.071155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.071155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:31.105184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:31.105566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:31.105687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:31.105893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:31.106573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:31.106948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.107771 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:31.108607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.108986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.109454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:31.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:31.110064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.110064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:31.110064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.110064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:31.110592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.111029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.111664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:31.117193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:31.117508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:31.117623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:31.117754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:31.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:31.118740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.119575 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:31.120413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.120934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.121829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:31.122251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:31.122629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.122629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:31.122629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.122629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.123665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:31.124166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:31.124808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.124808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:31.124808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.124808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:31.150276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:31.150776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:31.150975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:31.151277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:31.151969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:31.152293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:31.152546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.153703 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.155213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.156072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:31.156848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.156848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:31.157094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:31.158095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:31.159402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:31.159402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:31.159402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:35.722928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:35.723913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:35.723913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:35.724970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:35.759436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:35.759790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:35.760480 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:35.761139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:35.762183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:35.762711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:35.763354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:35.764135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:35.764135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:35.764135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:35.764135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:35.764135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:35.764135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:36.062197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:36.062821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:36.062821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.064363 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.065718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.066628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:36.067497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:36.068222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.068690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:36.068690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:36.068690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.068690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:36.068690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.068690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:36.105641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:36.105954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:36.105993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:36.106107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:36.106621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:36.106848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.107286 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:36.107856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.108155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.108598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:36.109114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:36.109810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.110114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.110543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:36.120252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:36.120614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:36.120729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:36.120885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:36.121255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:36.121477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.122028 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:36.122988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.123733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.124208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:36.124689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:36.125311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:36.126044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.127294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:36.150787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:36.151246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:36.151458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:36.151773 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:36.152467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:36.152934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:36.153176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.154262 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.156176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.156929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:36.157627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.157627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:36.158335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:36.159433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:36.160667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:36.160667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:36.160667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:40.723461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:40.724136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:40.724136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:40.724923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:40.759040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:40.759484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:40.760102 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:40.761056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:40.761791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:40.762512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:40.763419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:40.764581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:40.764581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:40.764581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:40.764581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:40.764581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:40.764581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:41.062993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:41.063673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:41.063814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:41.065093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:41.065476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:41.065867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.066718 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.067681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.068389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:41.068996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:41.069732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.070651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:41.070651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:41.070651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.070651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:41.070651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.070651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:41.106194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:41.107101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:41.107337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:41.107562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:41.108412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:41.108860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.109859 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:41.111306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.112143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:41.113405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:41.114041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.114041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:41.114041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.114041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:41.114461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.115269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.116115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:41.120786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:41.121166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:41.121286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:41.121426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:41.121876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:41.122125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.122702 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:41.123553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.123965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.124531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:41.124905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:41.125325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.125325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:41.125325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.125325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.125842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:41.127085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:41.128010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.128010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:41.128010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.128010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:41.151612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:41.152467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:41.152467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:41.152467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:41.153144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:41.153621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:41.153936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.154865 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.156343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.156715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:41.157410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.157410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:41.157990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:41.158455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:41.159035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:41.159035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:41.159035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:45.724426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:45.725275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:45.725275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:45.726362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:45.762345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:45.763161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:45.763892 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:45.764752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:45.766492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:45.767545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:45.768110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:45.768680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:45.768680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:45.768680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:45.768680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:45.768680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:45.768680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:46.064051 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:46.064286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:46.064477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:46.065527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:46.065963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:46.066233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.067204 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.068797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.069481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:46.070196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.070196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:46.070450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:46.071419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.072282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:46.072282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.072282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:46.106886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:46.107339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:46.107519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:46.107707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:46.108363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:46.108811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.109724 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:46.110998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.111480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.112297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:46.112721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:46.113137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.113137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:46.113137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.113137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:46.113925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.114372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.114911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:46.121221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:46.121476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:46.121590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:46.121722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:46.122092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:46.122352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.122915 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:46.123671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.124051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.125034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:46.125445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:46.125997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.125997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:46.125997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.125997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.126726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:46.127320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:46.128029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.128029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:46.128029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.128029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:46.151856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:46.152359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:46.152489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:46.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:46.153290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:46.153641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:46.153919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.154636 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.156054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.156922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:46.157733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:46.158788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:46.160064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:46.160064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:46.160064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:46.160064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:46.160064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:46.160064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:50.724943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:50.725720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:50.725832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:50.727327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:50.769045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:50.769564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:50.770784 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:50.771788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:50.772463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:50.773188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:50.774011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:50.775388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:50.775388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:50.775388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:50.775388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:50.775388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:50.775388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:51.064298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:51.065006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:51.065213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:51.066345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:51.066657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:51.066900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.067671 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.069336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.069960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:51.070279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.070279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:51.071335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:51.071807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.072315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:51.072315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.072315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:51.107520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:51.108201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:51.108201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:51.108201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:51.109055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:51.109194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.110372 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:51.111780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:51.112619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.113604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.114441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:51.115422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.116694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.117986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:51.121846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:51.122162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:51.122274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:51.122399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:51.122696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:51.122950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.123368 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:51.124096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.124532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.125567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:51.125934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:51.126301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.126301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:51.126301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.126301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.126712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:51.127254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:51.127745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.127745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:51.127745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.127745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:51.152521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:51.152951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:51.153135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:51.153389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:51.153777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:51.154209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:51.154356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.154932 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.156454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.157293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:51.158031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.158031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:51.158162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:51.158648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:51.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:51.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:51.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:55.725632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:55.726384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:55.726470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:55.727427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:55.758751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:55.759198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:55.760124 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:55.760901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:55.761630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:55.762349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:55.763203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:55.764291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:55.764291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:55.764291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:55.764291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:55.764291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:55.764291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:56.065144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:56.065690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:56.065849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:56.066967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:56.067221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:56.067384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.068251 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.069961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.070740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:56.071348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.071348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:56.071662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:56.072653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.073593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:56.073593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.073593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:56.108364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:56.108997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:56.109364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:56.109680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:51:56.110458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:56.110832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.111968 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:56.113310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.114132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.114566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:56.115168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:56.115817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.116546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.117940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:56.122297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:56.122522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:56.122660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:56.123258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:56.123258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:56.123589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.124295 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:51:56.125183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.125576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.127026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:56.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:56.127777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.127777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:56.127777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.127777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.129222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:56.129692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:56.130209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.130209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:56.130209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.130209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:51:56.153237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:51:56.153619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:51:56.153789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:51:56.154124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:51:56.154909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:56.155385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:51:56.155649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.156661 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.157962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.158558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:51:56.159134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.159134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:51:56.159241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:51:56.159958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:51:56.160984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:51:56.160984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:51:56.160984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:00.726391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:00.727269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:00.727269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:00.729248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:00.764659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:00.765074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:00.765961 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:00.766995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:00.767654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:00.768321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:00.769159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:00.770223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:00.770223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:00.770223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:00.770223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:00.770223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:00.770223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:01.065939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:01.066256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:01.066484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:01.067807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:01.068317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:01.068661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.069921 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.071185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.072094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:01.072980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:01.074210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.075504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:01.075504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:01.075504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.075504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:01.075504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.075504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:01.108750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:01.109432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:01.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:01.109947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:01.110968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:01.111373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.112202 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:01.113216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.113986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.114988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:01.115799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:01.116288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.116651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.117191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:01.122718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:01.123091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:01.123260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:01.123414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:01.123885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:01.124175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.124811 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:01.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.126435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.126942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:01.127631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:01.128392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:01.129089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.129985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:01.154001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:01.154463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:01.154680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:01.155057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:01.157937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:01.157937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:01.158141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.159211 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.161058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.162137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:01.162754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.162754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:01.163372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:01.164823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:01.165688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:01.165688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:01.165688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:05.727307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:05.748199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:05.748199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:05.749156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:05.786486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:05.787669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:05.787830 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:05.788527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:05.789118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:05.789535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:05.790029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:05.790641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:05.790641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:05.790641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:05.790641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:05.790641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:05.790641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:06.066862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:06.067217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:06.067563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:06.068512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:06.068852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:06.069091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.070533 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.072091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.072503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:06.072883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.072883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:06.073854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:06.074323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.074855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:06.074855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.074855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:06.109478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:06.110791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:06.111092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:06.111381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:06.112130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:06.112492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.113600 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:06.115114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.116331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.116945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:06.117665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:06.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:06.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.118278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:06.119068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.119815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.120712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:06.123172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:06.123588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:06.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:06.124174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:06.124560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:06.125077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.125559 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:06.126333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.127125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.127465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:06.127960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:06.128523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.128523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:06.128523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.128523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.128686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:06.129374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:06.130085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.130085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:06.130085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.130085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:06.154865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:06.155356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:06.155546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:06.155941 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:06.156477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:06.157063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:06.157369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.159060 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.160076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.160555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:06.161051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.161051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:06.161421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:06.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:06.162705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:06.162705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:06.162705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:10.748188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:10.749181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:10.749181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:10.750321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:10.785946 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:10.786345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:10.787555 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:10.788339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:10.789102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:10.790003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:10.790857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:10.792009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:10.792009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:10.792009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:10.792009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:10.792009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:10.792009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:11.067249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:11.067687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:11.067913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:11.069109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:11.069385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:11.069600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.070470 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.072937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.073547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:11.074202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.074202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:11.074934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:11.075738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.076644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:11.076644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.076644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:11.111033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:11.111592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:11.111774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:11.112023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:11.112613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:11.113024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.113988 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:11.115378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.116114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.116703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:11.117254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:11.117681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.117681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:11.117681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.117681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:11.118345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.118949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.119571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:11.123566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:11.123949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:11.124038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:11.124190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:11.124592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:11.125008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.125701 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:11.126824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.127178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.128009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:11.128311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:11.128617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.128617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:11.128617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.128617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.129389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:11.129832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:11.130297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.130297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:11.130297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.130297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:11.155394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:11.155990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:11.156183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:11.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:11.157036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:11.157408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:11.157605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.158501 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.160084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.160554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:11.161040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.161040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:11.161124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:11.161678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:11.162362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:11.162362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:11.162362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:15.748965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:15.750334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:15.750334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:15.751305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:15.785287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:15.785703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:15.786720 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:15.787826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:15.788983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:15.789727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:15.790849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:15.792004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:15.792004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:15.792004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:15.792004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:15.792004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:15.792004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:16.068024 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:16.068538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:16.068802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:16.070242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:16.070620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:16.070931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.073202 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.074931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:16.075867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.075867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:16.076436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:16.077101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.077667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:16.077667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.077667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:16.111674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:16.112280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:16.112538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:16.112844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:16.113616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:16.114080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.115051 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:16.116620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.117424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.118024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:16.118580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:16.119196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.119196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:16.119196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.119196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:16.119569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.120112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.120698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:16.124132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:16.124540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:16.124696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:16.124927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:16.125376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:16.125714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.126560 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:16.127891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.128675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.129251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:16.130019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:16.131371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.131371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:16.131371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.131371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:16.132882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:16.133564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.133564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:16.133564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.133564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:16.156088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:16.156966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:16.157194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:16.157534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:16.158375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:16.158924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:16.159213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.160627 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.162235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.162982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:16.163499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.163499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:16.164284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:16.165363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:16.166346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:16.166346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:16.166346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:20.749692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:20.750821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:20.750821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:20.751908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:20.753954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=239 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:52:20.786819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:20.787285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:20.788327 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:20.789583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:20.790327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:20.790938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:20.791608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:20.792698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:20.792698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:20.792698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:20.792698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:20.792698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:20.792698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:21.068713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:21.069151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:21.069310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:21.070319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:21.070673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:21.071304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.072170 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.073395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.074001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:21.074587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:21.075298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.076104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:21.076104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:21.076104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.076104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:21.076104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.076104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:21.112505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:21.112836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:21.113060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:21.113289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:21.114269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:21.114638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.115705 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:21.117102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.117826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.118678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:21.119262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:21.119898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.119898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:21.119898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.119898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:21.120622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.121532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.122219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:21.124623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:21.124886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:21.125007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:21.125343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:21.126086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:21.126734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.127852 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:21.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.129139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.129794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:21.130150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:21.130534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.130534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:21.130534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.130534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.131691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:21.132176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:21.132710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.132710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:21.132710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.132710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:21.157527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:21.158131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:21.158492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:21.158787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:21.159780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:21.160250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:21.160495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.161613 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.163137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:21.164134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:21.164587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:21.165174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:21.165174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:21.165174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:21.165174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:21.165174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:21.165174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:25.750471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:25.751670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:25.751670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:25.752178 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:25.785304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:25.785304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:25.785573 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:25.786348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:25.787374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:25.787803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:25.788539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:25.789173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:25.789173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:25.789173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:25.789173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:25.789173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:25.789173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:26.070159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:26.070159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:26.070159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:26.070857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:26.071292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:26.071794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.072511 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.073471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.073939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:26.074375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.074375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:26.074644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:26.075204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.075803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:26.075803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.075803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:26.113028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:26.113661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:26.113969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:26.114371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:26.115074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:26.115074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.115734 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:26.117988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:26.118854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.119567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.120247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:26.120747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.121497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.122422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:26.125027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:26.125684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:26.125684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:26.125684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:26.126018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:26.126589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.128954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:26.130014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.130687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.131307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:26.132641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:26.133115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.133807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:26.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:26.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:26.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:26.158962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:26.159354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:26.159649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:26.159815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.160380 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.161253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.161515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:26.161776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.161776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:26.162789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:26.163119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:26.163538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:26.163538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:26.163538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:30.751163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:30.752007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:30.752089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:30.753339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:30.791206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:30.791625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:30.792469 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:30.793271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:30.794021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:30.794678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:30.795590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:30.796635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:30.796635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:30.796635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:30.796635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:30.796635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:30.796635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:31.069859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:31.070472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:31.070684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:31.071935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:31.072304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:31.072548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.073459 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.075165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.075542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:31.075948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.075948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:31.077273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:31.078168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.079152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:31.079152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.079152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:31.113635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:31.113989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:31.114106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:31.114263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:31.115079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:31.115488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.116400 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:31.117641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.118528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.119019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:31.119627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:31.120280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.120280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:31.120280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.120280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:31.120655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.121409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.122239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:31.125604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:31.125887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:31.126085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:31.126141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:31.126725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:31.127020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.127560 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:31.128454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.128856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.129330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:31.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:31.130001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.130001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:31.130001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.130001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.130371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:31.130820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:31.131295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.131295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:31.131295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.131295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:31.158751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:31.159276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:31.159385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:31.159647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:31.160339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:31.160935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:31.161106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.161801 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.163631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.164273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:31.164844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.164844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:31.165851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:31.166555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:31.167656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:31.167656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:31.167656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:35.751953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:35.752985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:35.752985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:35.753956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:35.793404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:35.793927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:35.794998 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:35.796035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:35.796649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:35.797297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:35.798865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:35.800549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:35.800549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:35.800549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:35.800549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:35.800549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:35.800549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:36.070949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:36.071401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:36.071401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:36.072666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:36.073099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:36.073353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.074373 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.075230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.075630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:36.076025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:36.076986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.077739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:36.077739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:36.077739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.077739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:36.077739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.077739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:36.114205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:36.114840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:36.115064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:36.115263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:36.116227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:36.116553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.117507 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:36.119290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.120109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.120969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:36.122679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:36.123962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.124639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.125411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:36.125928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:36.126216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:36.126340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:36.126470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:36.127660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:36.128148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.129099 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:36.129947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.130397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.130937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:36.132081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:36.132498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:36.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.134172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:36.159290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:36.159794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:36.160111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:36.160218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:36.161187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:36.161630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:36.161923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.162910 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.164112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.164893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:36.165753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:36.166544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:36.167596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:36.167596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:36.167596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:36.167596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:36.167596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:36.167596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:40.753002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:40.753838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:40.753838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:40.754864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:40.791056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:40.791377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:40.792168 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:40.792853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:40.793455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:40.794145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:40.794868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:40.795846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:40.795846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:40.795846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:40.795846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:40.795846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:40.795846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:41.071674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:41.072153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:41.072417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:41.073601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:41.073979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:41.074219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.075064 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.076740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.077306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:41.077931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.077931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:41.079134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:41.079983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.080955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:41.080955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.080955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:41.114880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:41.115539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:41.115745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:41.115985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:41.116599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:41.117012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.117965 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:41.119366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.120470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.121105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:41.122245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:41.123267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.124349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.125716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:41.126548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:41.126783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:41.127020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:41.127241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:41.127871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:41.128305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.129248 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:41.130570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.131174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:41.132646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:41.133171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.133171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:41.133171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.133171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.134154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:41.134822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:41.135522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.135522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:41.135522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.135522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:41.160013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:41.160440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:41.160557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:41.160749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:41.161498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:41.162068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:41.162386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.163389 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.164431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.165242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:41.165907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:41.166623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:41.167603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:41.167603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:41.167603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:41.167603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:41.167603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:41.167603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:45.753656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:45.754523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:45.754726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:45.756334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:45.791934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:45.792438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:45.793505 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:45.794419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:45.795073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:45.795718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:45.796541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:45.797481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:45.797481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:45.797481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:45.797481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:45.797481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:45.797481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:46.072245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:46.072756 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:46.073045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:46.074346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:46.074706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:46.074992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.076060 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.077600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.078285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:46.078871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.078871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:46.079593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:46.080417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.081336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:46.081336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.081336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:46.115415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:46.115780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:46.115996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:46.116184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:46.117003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:46.117421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.118473 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:46.119651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.120177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:46.121383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:46.121894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.121894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:46.121894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.121894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:46.122612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.123256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.123807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:46.126839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:46.127171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:46.127355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:46.127562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:46.128134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:46.128522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.129373 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:46.130337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.130977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:46.131984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:46.132562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.132562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:46.132562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.132562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.133015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:46.133715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:46.134456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.134456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:46.134456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.134456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:46.160732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:46.161356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:46.161356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:46.161575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:46.162748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:46.163392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:46.163717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.164798 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.165799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.166612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:46.167336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:46.168247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:46.169310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:46.169310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:46.169310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:46.169310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:46.169310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:46.169310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:50.754449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:50.755526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:50.755526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:50.756607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:50.792803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:50.793237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:50.794249 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:50.795339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:50.795939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:50.796509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:50.797247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:50.798102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:50.798102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:50.798102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:50.798102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:50.798102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:50.798102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:51.072792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:51.073170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:51.073334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:51.074400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:51.074865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:51.075125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.076088 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.077204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.078001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:51.078579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:51.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.080481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:51.080481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:51.080481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.080481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:51.080481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.080481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:51.115898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:51.116269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:51.116452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:51.116642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:51.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:51.117632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.118527 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:51.119787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.120459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.121161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:51.121781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:51.122400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.122400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:51.122400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.122400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:51.122845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.123604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.124376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:51.127270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:51.127531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:51.127642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:51.127779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:51.128290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:51.128695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.129474 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:51.130216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.130635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.131080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:51.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:51.131823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.131823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:51.131823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.131823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.132127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:51.132606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:51.133154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.133154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:51.133154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.133154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:51.161399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:51.161717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:51.161934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:51.162184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:51.162725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:51.163167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:51.163449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.164288 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.165838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.166545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:51.166996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.166996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:51.167463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:51.167926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:51.168410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:51.168410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:51.168410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:55.755404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:55.756911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:55.756911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:55.757462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:55.796710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:55.797134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:55.798084 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:55.799036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:55.799971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:55.800674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:55.801634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:55.802700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:55.802700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:55.802700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:55.802700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:55.802700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:55.802700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:56.073453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:56.074124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:56.074315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:56.075508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:56.075859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:56.076118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.077140 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.078655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.079318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:56.079925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.079925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:56.080482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:56.081408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.082297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:56.082297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.082297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:56.116577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:56.117053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:56.117235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:56.117417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:52:56.118215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:56.118529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.119346 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:56.120615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.121503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.122139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:56.122832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:56.123435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.123435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:56.123435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.123435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:56.123852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.124585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.125571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:56.127738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:56.128131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:56.128304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:56.128497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:56.129077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:56.129470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.130225 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:52:56.130980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.131427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.131828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:56.132182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:56.132553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.132553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:56.132553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.132553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.132863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:56.133362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:56.133888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.133888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:56.133888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.133888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:52:56.161951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:52:56.162492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:52:56.162802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:52:56.163153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:52:56.163687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:56.164145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:52:56.164318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.165182 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.166124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.166694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:52:56.167287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:52:56.167912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:52:56.168810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:52:56.168810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:52:56.168810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:52:56.168810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:52:56.168810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:52:56.168810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:00.756209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:00.757375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:00.757375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:00.757730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:00.789894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:00.790139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:00.790855 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:00.791807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:00.792740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:00.793573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:00.794508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:00.795608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:00.795608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:00.795608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:00.795608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:00.795608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:00.795608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:01.074005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:01.074625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:01.074906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:01.076078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:01.076405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:01.076633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.077425 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.078456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.078983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:01.079679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.079679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:01.079816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:01.080402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.081115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:01.081115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.081115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:01.117288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:01.117753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:01.117968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:01.118182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:01.118960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:01.119313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.120348 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:01.121496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:01.122828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.124028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.124882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:01.125649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.126591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.127830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:01.128345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:01.128701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:01.128850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:01.128982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:01.129493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:01.129968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.130697 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:01.131446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.132019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.132397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:01.132858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:01.133345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.133345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:01.133345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.133345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.133538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:01.134197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:01.134900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.134900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:01.134900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.134900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:01.162441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:01.162721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:01.162878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:01.163037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:01.163603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:01.164153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:01.164397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.165145 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.166027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.166505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:01.166993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:01.167541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:01.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:01.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:01.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:01.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:01.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:01.168302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:05.756818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:05.757909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:05.757909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:05.758750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:05.792427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:05.792625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:05.792944 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:05.793608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:05.794273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:05.794940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:05.796669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:05.796669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:05.796669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:05.796669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:05.796669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:05.796669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:06.074693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:06.075638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:06.075871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:06.077403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:06.077943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:06.078250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.079245 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.080789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.081407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:06.082097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.082097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:06.082258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:06.083061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:06.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.084161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:06.117892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:06.118597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:06.118879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:06.119172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:06.120068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:06.120447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.121443 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:06.123353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.123820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.124563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:06.125579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:06.127308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.128403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:06.129805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:06.130997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:06.130997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:06.130997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:06.130997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.131476 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:06.132221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.132890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.133297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:06.133743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:06.134369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.134369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:06.134369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.134369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.135460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:06.135874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:06.136236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.136236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:06.136236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.136236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:06.163014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:06.163546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:06.163782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:06.164067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:06.164703 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:06.165120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:06.165399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.166285 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.167276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.167731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:06.168196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.168196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:06.168295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:06.168943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:06.169681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:06.169681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:06.169681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:10.757595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:10.758426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:10.758593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:10.760033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:10.804486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:10.804929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:10.805867 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:10.806947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:10.807635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:10.808275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:10.809004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:10.810263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:10.810263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:10.810263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:10.810263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:10.810263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:10.810263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:11.075431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:11.075997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:11.076253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:11.077390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:11.078732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:11.079127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.080035 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.081510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.082464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:11.083149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.083149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:11.083264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:11.084052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.084984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:11.084984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.084984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:11.119101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:11.120080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:11.120306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:11.120544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:11.121428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:11.121738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.122899 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:11.124467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.125878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:11.126677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:11.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:11.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.127422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:11.130445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.130989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:11.131693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:11.132064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:11.150250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:11.150465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.151157 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:11.151872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.021 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:11.152785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:11.153496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:11.154008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:11.154420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.155046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:11.163594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:11.163872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:11.163951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:11.164104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:11.164625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:11.164980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:11.165199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.165868 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.166472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.168181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:11.168733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:11.169294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:11.169914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:11.169914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:11.169914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:11.169914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:11.169914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:11.169914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:15.758687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:15.759717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:15.759717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:15.760195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:15.793924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:15.794357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:15.795307 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:15.796082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:15.796723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:15.797379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:15.798103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:15.799150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:15.799150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:15.799150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:15.799150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:15.799150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:15.799150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:16.076210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:16.076710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:16.076964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:16.078246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:16.078582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:16.078903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.080870 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.081941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.082614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:16.083211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:16.083974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.084867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:16.084867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:16.084867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.084867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:16.084867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.084867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:16.119933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:16.120400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:16.120580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:16.120775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:16.121666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:16.122068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.123032 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:16.124381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.125389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.126357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:16.127249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:16.128455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.129312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.130803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:16.131824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:16.131824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:16.131824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:16.131824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:16.131824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:16.132059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.132526 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:16.133334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.134005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.134378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:16.134846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:16.135372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:16.135970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.136681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:16.164415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:16.164855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:16.165049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:16.165348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:16.166074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:16.166525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:16.166851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.167833 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.169164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.169886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:16.170471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.170471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:16.170708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:16.171516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:16.172599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:16.172599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:16.172599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:20.759478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:20.760590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:20.760590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:20.761500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:20.763516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:53:20.800582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:20.800920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:20.801973 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:20.802786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:20.803556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:20.804270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:20.805093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:20.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:20.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:20.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:20.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:20.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:20.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:21.077068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:21.077699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:21.077929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:21.079514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:21.080011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:21.080349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.081422 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.083063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.083455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:21.083836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.083836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:21.084685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:21.085143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.085680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:21.085680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.085680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:21.120607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:21.120963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:21.121047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:21.121191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:21.121789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:21.122158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.122882 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:21.124091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.124930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.125375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:21.126176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:21.126953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.127666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.128715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:21.131397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:21.131790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:21.131904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:21.132054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:21.132465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:21.132852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.133547 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:21.134337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.134790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.135195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:21.135544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:21.135960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.135960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:21.135960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.135960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.136246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:21.136737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:21.137265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.137265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:21.137265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.137265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:21.182836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:21.182836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:21.182836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:21.182836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:21.182836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:21.182836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.183424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.184922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:21.185667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:21.186558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:21.187722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:25.760320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:25.761343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:25.761343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:25.761870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:25.798676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:25.799111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:25.799992 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:25.800863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:25.801502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:25.802244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:25.803566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:25.805604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:25.805604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:25.805604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:25.805604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:25.805604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:25.805604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:26.077837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:26.078423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:26.078630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:26.079953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:26.080285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:26.080528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.081428 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.083253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.084002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:26.084587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.084587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:26.085466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:26.086145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.086646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:26.086646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.086646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:26.121152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:26.121627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:26.121858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:26.122057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:26.122627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:26.123091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.123868 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:26.124746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:26.125836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.126396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.126905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:26.127431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.128528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.129487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:26.131837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:26.132430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:26.132749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:26.133102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:26.133904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:26.134463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.135299 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:26.136440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.136894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.137658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:26.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:26.138499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.138499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:26.138499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.138499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.139260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:26.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:26.140299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.140299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:26.140299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.140299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:26.165729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:26.166297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:26.166489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:26.166774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:26.167498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:26.167772 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:26.167944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.168640 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.170743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.171248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:26.171611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.171611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:26.173090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:26.173611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:26.174277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:26.174277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:26.174277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:30.760885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:30.762009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:30.762567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:30.763863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:30.806011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:30.806499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:30.807947 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:30.809427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:30.810124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:30.810753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:30.811411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:30.812159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:30.812159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:30.812159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:30.812159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:30.812159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:30.812159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:31.078648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:31.079126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:31.079283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:31.080211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:31.080578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:31.080861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.081783 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.082959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.083600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:31.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.084227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:31.084484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:31.085279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.086431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:31.086431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.086431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:31.121917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:31.122633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:31.122862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:31.123082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:31.123857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.124983 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:31.126311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.127540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.128172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:31.129031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:31.129909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.130825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.132035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:31.132507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:31.132507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:31.132615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:31.132750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:31.133186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:31.133422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.133982 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:31.134718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.135182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.135728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:31.136140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:31.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:31.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.136909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:31.137432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:31.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:31.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:31.166355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:31.166684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:31.166891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:31.167040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:31.167683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:31.168118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:31.168388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.169286 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.170290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.171213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:31.172067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:31.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:31.174330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:31.174330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:31.174330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:31.174330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:31.174330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:31.174330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:35.761726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:35.762581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:35.762581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:35.763087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:35.795201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:35.795572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:35.796460 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:35.797317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:35.798147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:35.799133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:35.800116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:35.801314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:35.801314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:35.801314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:35.801314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:35.801314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:35.801314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:36.079654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:36.080424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:36.080424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:36.081787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:36.081929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:36.082204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.083920 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.084596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.085494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:36.086356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:36.087012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.087513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:36.087513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:36.087513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.087513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:36.087513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.087513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:36.122503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:36.122960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:36.123134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:36.123319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:36.124009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:36.124347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.125217 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:36.126429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.126933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.127727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:36.128119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:36.128586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.128586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:36.128586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.128586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:36.129350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.129862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.130452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:36.132778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:36.133333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:36.133495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:36.133784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:36.134329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:36.134958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.135667 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:36.136398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.136957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.137259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:36.137723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:36.138217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.138217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:36.138217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.138217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.138373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:36.139065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:36.139750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.139750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:36.139750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.139750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:36.166962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:36.167321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:36.167445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:36.167606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:36.168215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:36.168645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:36.169025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.170091 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.171355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.171699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:36.172085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.172085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:36.172966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:36.173425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:36.173950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:36.173950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:36.173950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:40.762154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:40.762966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:40.762966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:40.763729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:40.805307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:40.805528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:40.806448 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:40.807412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:40.808085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:40.808865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:40.809916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:40.811139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:40.811139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:40.811139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:40.811139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:40.811139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:40.811139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:41.080270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:41.080963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:41.081157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:41.082247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:41.082609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:41.082965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.084054 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.085650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.086468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:41.087329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.087329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:41.087779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:41.088656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.090116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:41.090116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.090116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:41.123093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:41.123551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:41.123737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:41.123971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:41.124562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:41.124740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.125430 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:41.126710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.127535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.128071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:41.128672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:41.129314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.129314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:41.129314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.129314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:41.129657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.130544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.131658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:41.133437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:41.133697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:41.133794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:41.133949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:41.134560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:41.135077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.135910 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:41.136600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.137314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:41.138453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:41.139288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:41.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.140808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:41.167681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:41.168166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:41.168366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:41.168629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:41.169409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:41.169874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:41.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.171046 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.172594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.173150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:41.173614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.173614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:41.174336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:41.175000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:41.175739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:41.175739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:41.175739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:45.762681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:45.763326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:45.763326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:45.764181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:45.800203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:45.800203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:45.800480 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:45.801352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:45.802288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:45.802970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:45.802970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:45.804053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:45.805044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:45.805998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:45.805998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:45.805998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:46.081032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:46.081734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:46.082185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:46.083559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:46.084249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:46.084437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.085499 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.086661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.087059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:46.087409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.087409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:46.087815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:46.088251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.088748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:46.088748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.088748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:46.123696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:46.124333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:46.124526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:46.124706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:46.125222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:46.125508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.126171 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:46.127636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.128344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.129496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:46.130000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:46.130515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.130515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:46.130515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.130515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:46.130676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.131265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.131983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:46.133855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:46.134185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:46.134399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:46.134567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:46.135210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:46.135645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.136480 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:46.137524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.138104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.139060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:46.140117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:46.140911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:46.141912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.142929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:46.168400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:46.169198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:46.169388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:46.169636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:46.169968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:46.170163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:46.170266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.170735 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.171692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.172353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:46.173035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:46.173849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:46.174891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:46.174891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:46.174891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:46.174891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:46.174891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:46.174891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:50.763467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:50.764638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:50.764638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:50.765805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:50.800451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:50.800865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:50.801663 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:50.802612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:50.803568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:50.804122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:50.805041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:50.806148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:50.806148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:50.806148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:50.806148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:50.806148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:50.806148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:51.081883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:51.082204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:51.082375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:51.083254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:51.083553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:51.083961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.085047 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.086835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.087746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:51.088598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.088598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:51.089282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:51.089972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.090799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:51.090799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.090799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:51.124517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:51.125101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:51.125373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:51.125659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:51.126211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:51.126519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.127326 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:51.129059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.130109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.131240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:51.132948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:51.134111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.135172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:51.135696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:51.136586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:51.136586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:51.136586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:51.136586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.136960 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:51.138340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.139712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:51.139984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:51.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:51.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.140240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.141297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:51.141653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:51.142032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.142032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:51.142032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.142032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:51.169021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:51.169441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:51.169614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:51.169919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:51.170361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:51.170715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:51.170986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.171914 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.173251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.173582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:51.173942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.173942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:51.175067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:51.176058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:51.177038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:51.177038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:51.177038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:55.764262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:55.765458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:55.765458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:55.767219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:55.804668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:55.805116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:55.805997 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:55.807084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:55.807679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:55.808296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:55.808296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:55.808541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:55.809067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:55.809576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:55.809576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:55.809576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:56.082745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:56.083715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:56.083980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:56.085275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:56.085707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:56.086050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.087260 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.088593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.089386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:56.090114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:56.090953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.091947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:56.091947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:56.091947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.091947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:56.091947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.091947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:56.125338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:56.126526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:56.126526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:56.126734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:53:56.127456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:56.128122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.129396 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:56.130880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.131750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.132232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:56.132990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:56.133635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.133635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:56.133635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.133635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:56.133931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.134817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:56.135637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:56.136426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:56.136426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:56.136426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:56.136899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:56.137317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.138269 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:53:56.139154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.139585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.140065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:56.140454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:56.140852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.140852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:56.140852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.140852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.141499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:56.142083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:56.142641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.142641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:56.142641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.142641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:53:56.169822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:53:56.170709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:53:56.170970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:53:56.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:53:56.172157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:56.172630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:53:56.172974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.173948 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.175709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.176557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:53:56.177252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.177252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:53:56.177393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:53:56.178268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:53:56.179677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:53:56.179677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:53:56.179677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:00.764889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:00.766214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:00.766214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:00.767053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:00.809266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:00.809611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:00.810584 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:00.811586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:00.812286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:00.812713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:00.813194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:00.813745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:00.813745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:00.813745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:00.813745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:00.813745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:00.813745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:01.083658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:01.083979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:01.084155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:01.085399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:01.085857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:01.086175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.087155 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.088729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.089298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:01.089871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.089871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:01.090473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:01.091233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.092122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:01.092122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.092122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:01.125893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:01.126245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:01.126413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:01.126599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:01.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:01.127522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.128435 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:01.129560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.130347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.130839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:01.131463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:01.132211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.132211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:01.132211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.132211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:01.132435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.132921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.133374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:01.135911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:01.136195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:01.136308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:01.136434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:01.137000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:01.137434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.138276 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:01.139086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.139589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.139897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:01.140232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:01.140681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.140681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:01.140681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.140681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.140905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:01.141382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:01.141921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.141921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:01.141921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.141921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:01.170416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:01.170866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:01.171038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:01.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:01.171986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:01.172466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:01.172727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.173617 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.174936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.175517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:01.176309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.176309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:01.176492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:01.177244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:01.178097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:01.178097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:01.178097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:05.765891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:05.766650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:05.766650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:05.766948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:05.795684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:05.795950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:05.796522 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:05.797518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:05.797770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:05.798005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:05.798005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:05.798473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:05.798788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:05.799127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:05.799127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:05.799127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:06.084303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:06.084700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:06.084864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:06.085848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:06.086192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:06.086470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.087358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.088500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.089154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:06.089710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:06.090394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.090946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:06.090946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:06.090946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.090946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:06.090946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.090946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:06.126361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:06.126872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:06.127083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:06.127282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:06.128386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:06.128664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.129639 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:06.131029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.132285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.132939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:06.133504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:06.134020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.134518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.135199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:06.136374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:06.136624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:06.136722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:06.136890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:06.137355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:06.137780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.138583 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:06.139374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.139879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.140226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:06.140520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:06.140975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.140975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:06.140975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.140975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.141311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:06.141986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:06.142713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.142713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:06.142713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.142713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:06.170870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:06.171325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:06.171509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:06.171783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:06.172389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:06.172776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:06.173081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.173973 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.175548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:06.176288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.176288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:06.177551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:06.178006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:06.178500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:06.178500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:06.178500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:10.766610 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:10.767736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:10.767736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:10.768918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:10.803517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:10.804016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:10.804922 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:10.805648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:10.806268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:10.806927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:10.807624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:10.808567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:10.808567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:10.808567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:10.808567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:10.808567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:10.808567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:11.085334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:11.086050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:11.086050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:11.098172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.099528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.101471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:11.102963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:11.104026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.105667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:11.105667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.105667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:11.126963 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:11.127494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:11.127706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:11.127971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:11.128666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:11.129004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.129710 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:11.131149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.133056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:11.133475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:11.133902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.133902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:11.133902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.133902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:11.135256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.135809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.136401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:11.136823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:11.137100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:11.137226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:11.137382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:11.137711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:11.137973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.138506 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:11.139559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.140116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.140789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:11.141245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:11.141681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.141681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:11.141681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.141681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.142023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:11.142626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:11.143235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.143235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:11.143235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.143235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:11.171569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:11.172089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:11.172209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:11.172384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:11.173133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:11.173524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:11.173843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.175576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.176536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:11.177466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:11.178525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:11.179871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:11.179871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:11.179871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:11.179871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:11.179871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:11.179871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:15.767693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:15.768738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:15.768738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:15.769823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:15.805457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:15.805960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:15.806619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:15.808695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:15.809269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:15.809630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:15.809630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:15.810681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:15.811178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:15.811698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:15.811698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:15.811698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:16.085992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:16.086616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:16.086921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:16.088556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:16.089048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:16.089367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.090583 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.092516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.093105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:16.093670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.093670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:16.094806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:16.095518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.096076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:16.096076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.096076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:16.127656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:16.128415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:16.128612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:16.128860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:16.129561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:16.129948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.130909 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:16.132445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.133104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.133900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:16.134529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:16.135105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.135105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:16.135105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.135105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:16.135747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.136445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.136946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:16.137214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:16.137657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:16.137901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:16.138139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:16.138743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:16.139206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.140206 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:16.141549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.142024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.142983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:16.143325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:16.143819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.143819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:16.143819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.143819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.144746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:16.145205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:16.145702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.145702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:16.145702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.145702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:16.172187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:16.172666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:16.172798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:16.173025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:16.173612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:16.174113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:16.174379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.175214 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.176258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.176596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:16.176938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.176938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:16.177342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:16.177727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:16.178225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:16.178225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:16.178225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:20.768210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:20.768894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:20.768951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:20.769736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:20.771466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:54:20.806442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:20.806655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:20.807435 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:20.808358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:20.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:20.809170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:20.809170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:20.809485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:20.810470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:20.811408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:20.811408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:20.811408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:21.086585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:21.087107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:21.087292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:21.088278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:21.088563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:21.088796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.089689 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.091253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.093302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:21.094190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:21.095167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.097653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:21.097653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:21.097653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.097653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:21.097653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.097653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:21.128244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:21.128993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:21.128993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:21.129286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:21.129754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:21.130074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.131264 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:21.132489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.133284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.134333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:21.135100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:21.135726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.136454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.137363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:21.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:21.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:21.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:21.138098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:21.138353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:21.138557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.139035 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:21.139728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.140450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.140895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:21.141510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:21.142394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:21.142990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.143508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:21.172722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:21.173126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:21.173211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:21.173413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:21.173902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:21.174278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:21.174494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.175315 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.176594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.177249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:21.177895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.177895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:21.178112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:21.179059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:21.179933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:21.179933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:21.179933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:25.769072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:25.769682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:25.769959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:25.771570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:25.807174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:25.807650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:25.808399 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:25.809153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:25.809878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:25.810672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:25.811652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:25.812993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:25.812993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:25.812993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:25.812993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:25.812993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:25.812993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:26.087435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:26.087882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:26.088200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:26.089339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:26.089690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:26.089973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.090934 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.092574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.093027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:26.093443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.093443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:26.094587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:26.095100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.095637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:26.095637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.095637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:26.128967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:26.129328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:26.129552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:26.129789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:26.130506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:26.130912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.131876 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:26.133062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.134002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.134541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:26.135215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:26.135896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.135896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:26.135896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.135896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:26.136286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.136994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.137594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:26.137994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:26.138361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:26.138550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:26.138748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:26.139205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:26.139500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.140543 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:26.141331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.141797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.142337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:26.142716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:26.143132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.143132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:26.143132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.143132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.143380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:26.143923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:26.144549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.144549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:26.144549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.144549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:26.173095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:26.173358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:26.173858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:26.173858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:26.174227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:26.174634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:26.174984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.175712 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.176619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.177285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:26.178042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:26.178702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:26.179526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:26.179526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:26.179526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:26.179526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:26.179526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:26.179526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:30.769806 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:30.770725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:30.770725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:30.771684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:30.819101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:30.819349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:30.820212 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:30.821302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:30.822088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:30.822996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:30.824073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:30.825250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:30.825250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:30.825250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:30.825250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:30.825250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:30.825250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:31.088274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:31.088712 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:31.088893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:31.089791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:31.090176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:31.090513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.091342 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.092170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.092988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:31.093871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:31.095001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.096338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:31.096338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:31.096338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.096338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:31.096338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.096338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:31.129676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:31.130074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:31.130152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:31.130246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:31.130883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:31.131214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.132028 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:31.132685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:31.133557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.134344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.134969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:31.135486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.136234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.137023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:31.138490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:31.138741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:31.138833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:31.138934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:31.139411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:31.139896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.140597 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:31.141188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.141616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.141884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:31.142228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:31.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:31.143057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.143614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:31.173621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:31.173921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:31.174010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:31.174148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:31.174605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:31.175087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:31.175400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.176242 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.177611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.178381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:31.179019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.179019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:31.179201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:31.180158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:31.180848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:31.180848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:31.180848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:35.770668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:35.771668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:35.771668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:35.772734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:35.810030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:35.810030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:35.810813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:35.811773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:35.812585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:35.813297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:35.814322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:35.815914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:35.815914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:35.815914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:35.815914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:35.815914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:35.815914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:36.089128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:36.089695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:36.089938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:36.091208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:36.091427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:36.091736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.092397 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.094012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.094658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:36.095038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.095038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:36.095590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:36.096131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.096677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:36.096677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.096677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:36.130292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:36.130846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:36.131055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:36.131215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:36.131724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:36.132008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.132708 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:36.133910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.135231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:36.135590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:36.135943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.135943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:36.135943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.135943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:36.136969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.137415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.137873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:36.138987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:36.139456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:36.139696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:36.139939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:36.140388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:36.140780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.141575 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:36.142332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.143076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.143378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:36.143839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:36.144288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.144288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:36.144288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.144288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.144441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:36.145082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:36.145840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.145840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:36.145840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.145840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:36.174110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:36.174550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:36.174683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:36.174908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:36.175406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:36.175784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:36.176112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.176955 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.179296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:36.179781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.179781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:36.180151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:36.180820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:36.181510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:36.181510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:36.181510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:40.771187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:40.771966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:40.772091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:40.773474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:40.808223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:40.808223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:40.808749 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:40.809822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:40.810460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:40.811226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:40.812012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:40.813230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:40.813230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:40.813230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:40.813230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:40.813230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:40.813230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:41.089911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:41.090441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:41.090624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:41.092029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:41.092219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:41.092400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.093379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.094584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.095249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:41.096170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:41.097156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.098397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:41.098397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:41.098397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.098397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:41.098397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.098397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:41.131058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:41.131585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:41.131791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:41.132036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:41.132612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:41.132969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.133800 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:41.135236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.136079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.136859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:41.137452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:41.138063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.138063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:41.138063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.138063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:41.139051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.139786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:41.140286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:41.140605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:41.140749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:41.141164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.141893 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:41.142709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:41.144619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:41.145006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:41.145338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:41.145667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.146156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:41.174672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:41.175256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:41.175386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:41.175593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:41.176194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:41.176628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:41.176994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.177822 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.179435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.180110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:41.180806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:41.181501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:41.182463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:41.182463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:41.182463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:41.182463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:41.182463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:41.182463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:45.771781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:45.772471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:45.772876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:45.774471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:45.813252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:45.813417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:45.814142 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:45.815152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:45.815751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:45.816401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:45.817292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:45.818241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:45.818241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:45.818241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:45.818241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:45.818241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:45.818241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:46.090707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:46.091142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:46.091352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:46.092569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:46.092967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:46.093226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.094161 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.095977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.096645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:46.097318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.097318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:46.098096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:46.098713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:46.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.099345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:46.131686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:46.132384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:46.132652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:46.132931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:46.133583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:46.133962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.134797 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:46.136422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.137121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.138011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:46.138396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:46.138744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.138744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:46.138744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.138744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:46.140025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.140470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:46.140959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:46.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:46.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:46.141669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:46.141993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.142592 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:46.143830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.144263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.145407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:46.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:46.146329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.146329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:46.146329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.146329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.147507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:46.148197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:46.148794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.148794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:46.148794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.148794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:46.175196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:46.175622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:46.175938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:46.176235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:46.176917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:46.177347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:46.177667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.178591 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.179623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.180408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:46.181219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:46.182209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:46.183947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:46.183947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:46.183947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:46.183947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:46.183947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:46.183947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:50.772516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:50.773452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:50.773452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:50.774149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:50.816487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:50.816871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:50.817614 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:50.818688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:50.819910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:50.820316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:50.820818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:50.821396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:50.821396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:50.821396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:50.821396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:50.821396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:50.821396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:51.091363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:51.091942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:51.092240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:51.093181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:51.093548 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:51.093938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.095706 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.097557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.098018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:51.098362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.098362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:51.099583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:51.100112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.100778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:51.100778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.100778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:51.132513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:51.133033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:51.133146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:51.133266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:51.134106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:51.134508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.135480 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:51.136598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.137484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.138300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:51.139028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:51.139627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.139627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:51.139627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.139627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:51.140115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.140916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:51.141816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:51.142293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:51.142293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:51.142584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.143494 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:51.144786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.145217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.146495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:51.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:51.147241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.147241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:51.147241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.147241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.147681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:51.148186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:51.149020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.149020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:51.149020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.149020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:51.175879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:51.176408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:51.176637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:51.177314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:51.177744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:51.178217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:51.178484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.179457 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.180442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.181070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:51.181723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:51.182675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:51.183949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:51.183949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:51.183949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:51.183949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:51.183949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:51.183949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:55.773420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:55.776133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:55.776133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:55.776286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:55.823398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:55.823855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:55.824786 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:55.826265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:55.826964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:55.827628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:55.828433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:55.829476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:55.829476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:55.829476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:55.829476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:55.829476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:55.829476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:56.092482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:56.093112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:56.093335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:56.094208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:56.094428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:56.094542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.095255 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.096444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.097072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:56.097472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.097472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:56.098364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:56.099023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.099585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:56.099585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.099585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:56.133503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:56.133967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:56.134173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:56.134506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:54:56.135090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:56.135393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.136289 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:56.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.138828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.139281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:56.139929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:56.140590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.140590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:56.140590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.140590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:56.140823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.141584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:56.142537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:56.142960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:56.142960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:56.142960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.143395 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:54:56.144159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.144554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.145307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:56.145639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:56.146039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.146039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:56.146039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.146039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.146476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:56.147006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:56.147687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.147687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:56.147687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.147687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:54:56.176690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:54:56.177152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:54:56.177332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:54:56.177594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:54:56.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:56.178584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:54:56.178856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.179732 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.181047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.181872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:54:56.182540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.182540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:54:56.182714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:54:56.183606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:54:56.184554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:54:56.184554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:54:56.184554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:00.774386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:00.775218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:00.775218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:00.776487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:00.815370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:00.815895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:00.817126 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:00.818191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:00.818863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:00.819435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:00.820255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:00.821232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:00.821232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:00.821232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:00.821232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:00.821232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:00.821232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:01.093225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:01.094082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:01.094082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:01.095177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:01.095542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:01.095923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.096954 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.098665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.099238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:01.099658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.099658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:01.100677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:01.101260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.101886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:01.101886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.101886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:01.134018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:01.134796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:01.135046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:01.135278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:01.135816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:01.136038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.136900 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:01.138460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.139356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.140132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:01.140751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:01.141433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.141433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:01.141433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.141433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:01.142958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.143701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:01.144656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:01.145315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.145527 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:01.146045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:01.146880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:01.147593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:01.148253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:01.148870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.149943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:01.177188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:01.177916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:01.178166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:01.178518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:01.179135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:01.179553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:01.179890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.180725 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.182298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.182920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:01.183398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.183398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:01.184014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:01.184660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:01.185339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:01.185339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:01.185339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:05.775102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:05.776319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:05.776524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:05.777875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:05.820180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:05.820404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:05.821027 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:05.821937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:05.822348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:05.822741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:05.822741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:05.822935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:05.823400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:05.824126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:05.824126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:05.824126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:06.093802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:06.094103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:06.094103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:06.094789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:06.095034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:06.095186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.095955 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.097489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.097906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:06.098278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.098278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:06.099152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:06.099642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.100207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:06.100207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.100207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:06.134727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:06.135282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:06.135473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:06.135682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:06.136525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:06.136898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.137602 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:06.138541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:06.139611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.140537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.141365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:06.142169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.142963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:06.144083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:06.144608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:06.144608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:06.144608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:06.144930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:06.145009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.145592 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:06.146526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.146934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.147471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:06.147868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:06.148256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.148256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:06.148256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.148256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.148653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:06.149186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:06.149820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.149820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:06.149820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.149820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:06.177777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:06.178191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:06.178371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:06.178632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:06.180137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:06.180366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:06.180482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.181056 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.182466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.183255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:06.184031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:06.185028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:06.186062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:06.186062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:06.186062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:06.186062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:06.186062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:06.186062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:10.776553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:10.777193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:10.777193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:10.778038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:10.812689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:10.813265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:10.814666 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:10.815845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:10.816501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:10.817163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:10.818027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:10.819222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:10.819222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:10.819222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:10.819222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:10.819222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:10.819222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:11.094947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:11.094947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:11.095204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:11.096371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:11.096850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:11.097139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.098147 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.099795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.100182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:11.100561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.100561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:11.101041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:11.101684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.103211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:11.103211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.103211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:11.135454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:11.135935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:11.136126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:11.136337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:11.137361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:11.137740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.138733 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:11.140349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.141043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.142289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:11.142804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:11.143280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.143280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:11.143280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.143280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:11.144341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.144952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:11.145545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:11.145927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:11.146043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:11.146348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.147001 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:11.147779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.148239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.148706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:11.149091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:11.149519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.149519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:11.149519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.149519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.149815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:11.150433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:11.150999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.150999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:11.150999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.150999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:11.178482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:11.178800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:11.178942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:11.179149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:11.179866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:11.180462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:11.180917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.181705 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.182751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.183225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:11.183651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.183651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:11.183728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:11.184287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:11.185226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:11.185226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:11.185226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:15.777343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:15.778374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:15.778374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:15.779425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:15.816145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:15.816515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:15.817365 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:15.818243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:15.818932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:15.819538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:15.820224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:15.821026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:15.821026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:15.821026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:15.821026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:15.821026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:15.821026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:16.095166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:16.095495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:16.095692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:16.097222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:16.097685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:16.098012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.099370 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.101425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.102227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:16.103004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:16.103903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.104921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:16.104921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:16.104921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.104921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:16.104921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.104921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:16.136195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:16.136665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:16.137072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:16.137373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:16.138190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:16.138535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.139535 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:16.141229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.141683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.142463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:16.142845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:16.143286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.143286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:16.143286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.143286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:16.143925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.144398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:16.145190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:16.145427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:16.145427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:16.145491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:16.146067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:16.146490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.147262 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:16.148212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.148847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.149272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:16.149696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:16.150286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.150286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:16.150286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.150286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.150508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:16.151211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:16.151991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.151991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:16.151991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.151991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:16.179200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:16.179528 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:16.179643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:16.179860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:16.180454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:16.180879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:16.181161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.181937 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.183059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.183562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:16.184092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.184092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:16.184498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:16.185095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:16.185612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:16.185612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:16.185612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:20.777940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:20.778633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:20.779104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:20.780413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:20.782330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:55:20.817533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:20.817920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:20.818723 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:20.819413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:20.820048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:20.820620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:20.821265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:20.822076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:20.822076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:20.822076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:20.822076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:20.822076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:20.822076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:21.095753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:21.096278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:21.096401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:21.097951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:21.097951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:21.097951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.098651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.099876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.100517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:21.101137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:21.101733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.102302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:21.102302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:21.102302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.102302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:21.102302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.102302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:21.137032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:21.137549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:21.137735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:21.137980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:21.138816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:21.138816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.139002 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:21.140180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.140500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.141524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:21.141800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:21.142106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.142106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:21.142106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.142106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:21.143296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.143619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.143997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:21.145393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:21.145817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:21.145996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:21.146174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:21.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:21.146871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.146984 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:21.147721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.148057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.148644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:21.149310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:21.149594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.149594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:21.149594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.149594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.149968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:21.150281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:21.150671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.150671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:21.150671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.150671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:21.179686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:21.180278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:21.180471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:21.180746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:21.181911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:21.181911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:21.181911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.183705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.184117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:21.184474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.184474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:21.185499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:21.186146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:21.186642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:21.186642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:21.186642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:25.778522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:25.779404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:25.780453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:25.780453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:25.812608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:25.813031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:25.813884 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:25.815072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:25.815753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:25.816521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:25.816521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:25.816646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:25.817244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:25.817895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:25.817895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:25.817895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:26.096456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:26.097169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:26.097379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:26.098589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:26.099005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:26.099278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.100298 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.102028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.102711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:26.103375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.103375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:26.103485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:26.104286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.105220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:26.105220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.105220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:26.139188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:26.139483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:26.139708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:26.139973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:26.140593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:26.141009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.141929 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:26.143476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.144376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.145015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:26.145669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:26.146387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.146387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:26.146387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.146387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:26.146943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.147727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:26.148656 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:26.149169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:26.149169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:26.149169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.149373 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:26.150138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.150873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.151125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:26.151459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:26.151864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.151864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:26.151864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.151864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.152085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:26.152557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:26.153075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.153075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:26.153075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.153075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:26.180237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:26.180604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:26.180785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:26.181054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:26.181844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:26.182409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:26.182774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.183751 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.185276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.185928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:26.186495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.186495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:26.187169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:26.188007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:26.188945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:26.188945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:26.188945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:30.298895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:30.299566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:30.299812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:30.314668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 01:55:30.314786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 01:55:30.314875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:55:30.347545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.347927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.348254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.348496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.349158 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.350454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.351182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.351965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.351965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.351965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.351965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:55:30.382581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.382861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.383066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.383144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.383832 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.385143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.386191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.386697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.386697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.386697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.386697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:55:30.422448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.422747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.422905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.423005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.423584 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.426327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.426888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.427833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.427833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.427833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.427833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:55:30.457961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.458387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.459101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.459239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.460142 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.461429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.462028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.462651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.462651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.462651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.462651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:55:30.542403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.542720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.543205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.543339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.543567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.544318 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.544721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 01:55:30.545668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.546268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.546892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 01:55:30.643974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.644248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.644561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.644779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.645423 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.646277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.646883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.647574 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.346 seconds
May 05 01:55:30.779457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:30.779826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:30.780118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:30.781390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:30.815704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:30.816089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.816895 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.817872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.818321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:30.818704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.818704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:30.818984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:30.819693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:30.820325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:30.820325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:30.820325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:31.097038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:31.097836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:31.098008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:31.099113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:31.099469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:31.099719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.100624 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.102271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.102778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:31.103243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.103243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:31.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:31.104653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.105608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:31.105608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.105608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:31.139538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:31.139922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:31.140129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:31.140302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:31.140984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:31.141290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.142259 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:31.143552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.144457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.145050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:31.145712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:31.146390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.146390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:31.146390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.146390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:31.146778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.147553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:31.148435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:31.148960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:31.148960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:31.149142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.149933 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:31.150609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.151107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.151505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:31.151890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:31.152413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.152413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:31.152413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.152413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.152544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:31.153129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:31.153785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.153785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:31.153785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.153785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:31.180918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:31.181514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:31.181704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:31.182021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:31.182655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:31.183229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:31.183445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.184377 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.186142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.186658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:31.187045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.187045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:31.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:31.187704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:31.188253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:31.188253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:31.188253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:35.780181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:35.781037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:35.781037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:35.782502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:35.817851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:35.818264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:35.819250 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:35.820014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:35.821058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:35.821752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:35.822590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:35.823752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:35.823752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:35.823752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:35.823752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:35.823752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:35.823752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:36.097750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:36.098225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:36.098441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:36.099694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:36.100101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:36.100354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.101171 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.102426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.102990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:36.103518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:36.104147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.104934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:36.104934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:36.104934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.104934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:36.104934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.104934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:36.139950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:36.140234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:36.140326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:36.140457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:36.141189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:36.141452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.142211 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:36.143251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.143978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.144403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:36.145014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:36.145633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.145633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:36.145633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.145633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:36.145877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.146584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.147430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:36.147929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:36.148223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:36.148382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:36.148556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:36.148882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:36.149098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.149670 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:36.150482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.150911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.151483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:36.151831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:36.152215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.152215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:36.152215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.152215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.152620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:36.153111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:36.153613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.153613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:36.153613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.153613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:36.181384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:36.181666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:36.181831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:36.182056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:36.182954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:36.183497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:36.183853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.184856 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.185885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.186429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:36.187001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.187001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:36.187207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:36.187928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:36.188852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:36.188852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:36.188852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:40.780796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:40.781404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:40.781597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:40.782919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:40.819637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:40.820050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:40.820999 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:40.822012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:40.822647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:40.823241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:40.824322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:40.826118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:40.826118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:40.826118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:40.826118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:40.826118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:40.826118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:41.098347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:41.099009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:41.099261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:41.100010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:41.100190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:41.100384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.101042 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.102079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.102735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:41.103188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.103188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:41.103565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:41.104170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.104643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:41.104643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.104643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:41.140902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:41.141689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:41.142073 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:41.142362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:41.143315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:41.143645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.144612 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:41.146240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.147988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.148876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:41.149580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:41.151447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.152523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:41.153217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:41.153606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:41.153606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:41.153606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:41.153606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.154152 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:41.155125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.156461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:41.157601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:41.158211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:41.158834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.160200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:41.182208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:41.183070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:41.183288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:41.183576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:41.184290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:41.184796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:41.185208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.186040 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.187695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:41.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:41.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:41.191287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:41.192875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:41.192875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:41.192875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:45.781556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:45.782717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:45.782717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:45.782717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:45.812920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:45.813267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:45.814143 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:45.814912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:45.815521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:45.816171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:45.816901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:45.817732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:45.817732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:45.817732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:45.817732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:45.817732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:45.817732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:46.098967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:46.099513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:46.099675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:46.100554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:46.100994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:46.101321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.102518 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.105491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:46.106096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:46.106950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.107709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:46.107709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:46.107709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.107709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:46.107709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.107709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:46.141530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:46.142137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:46.142348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:46.142569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:46.143189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:46.143491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.144434 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:46.145941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.147338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.147893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:46.149056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:46.150356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.151344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.152064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:46.152501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:46.152501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:46.152604 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:46.153010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:46.153110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:46.153369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.153918 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:46.154659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.155276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.155640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:46.155986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:46.156283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.156283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:46.156283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.156283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.156836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:46.157167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:46.157500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.157500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:46.157500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.157500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:46.182776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:46.183131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:46.183260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:46.183430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:46.183775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:46.184048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:46.184197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.184941 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.185864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.186126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:46.186352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.186352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:46.186876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:46.187195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:46.187530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:46.187530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:46.187530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:50.782140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:50.783171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:50.783171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:50.783878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:50.821731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:50.822238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:50.823359 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:50.824602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:50.825361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:50.826153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:50.827103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:50.828191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:50.828191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:50.828191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:50.828191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:50.828191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:50.828191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:51.099704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:51.100197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:51.100272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:51.101687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:51.102029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:51.102210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.103085 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.104870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:51.105128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.105128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:51.105987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:51.106326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.106722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:51.106722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.106722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:51.142283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:51.142924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:51.143063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:51.143212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:51.144231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:51.144605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.145625 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:51.146644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.147492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.148029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:51.148620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:51.149274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.150295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.151163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:51.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:51.153213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:51.153378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:51.153588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:51.154171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:51.154527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.156093 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:51.156455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.156892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.157609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:51.157984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:51.158388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.158388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:51.158388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.158388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.159113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:51.159739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:51.160411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.160411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:51.160411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.160411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:51.183345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:51.183686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:51.183815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:51.184026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:51.184611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:51.185061 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:51.185336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.186161 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.187220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.187907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:51.188530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.188530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:51.188672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:51.189398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:51.190382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:51.190382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:51.190382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:55.784906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:55.784906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:55.784906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:55.786434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:55.827987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:55.828329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:55.829224 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:55.830124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:55.830863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:55.831549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:55.832339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:55.833384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:55.833384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:55.833384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:55.833384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:55.833384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:55.833384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:56.100632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:56.101114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:56.101385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:56.102491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:56.102874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:56.103129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.104064 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.105711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.106389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:56.107099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.107099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:56.109120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:56.110520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.111754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:56.111754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.111754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:56.143733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:56.144338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:56.144534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:56.144747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:55:56.145366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:56.145685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.146632 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:56.148020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.148877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.149728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:56.151087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:56.151807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.152479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:56.154202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:56.154819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:56.154819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:56.154819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:56.154819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.155359 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:55:56.156095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.156542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.157067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:56.157474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:56.158004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.158004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:56.158004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.158004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.158518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:56.159145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:56.159707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.159707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:56.159707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.159707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:55:56.183979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:55:56.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:55:56.184589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:55:56.184873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:55:56.185315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:56.185707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:55:56.186046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.186804 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.188083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.188710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:55:56.189314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.189314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:55:56.189634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:55:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:55:56.191395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:55:56.191395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:55:56.191395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:00.784126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:00.784472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:00.784578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:00.785509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:00.817891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:00.818172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:00.818873 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:00.819651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:00.820089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:00.820510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:00.821010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:00.821577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:00.821577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:00.821577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:00.821577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:00.821577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:00.821577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:01.101263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:01.101558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:01.101686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:01.102645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:01.102948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:01.103198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.104497 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.106238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.107105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:01.107865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.107865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:01.108068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:01.108967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.110037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:01.110037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.110037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:01.144359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:01.144706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:01.144856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:01.145019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:01.145613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:01.146013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.146796 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:01.147631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:01.148540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.149361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:01.150643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.151439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.152319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:01.153659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:01.153978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:01.154089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:01.154228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:01.154655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:01.155053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.155746 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:01.156585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.157133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.157551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:01.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:01.158271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.158271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:01.158271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.158271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.158844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:01.159307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:01.159815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.159815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:01.159815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.159815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:01.184531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:01.185009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:01.185207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:01.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:01.186000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:01.186478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:01.186718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.187524 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.188862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.189862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:01.190510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.190510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:01.190843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:01.191459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:01.192255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:01.192255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:01.192255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:05.797100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:05.797100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:05.797100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:05.797100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:05.829443 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:05.829619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:05.830174 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:05.830973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:05.831805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:05.832470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:05.833400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:05.834578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:05.834578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:05.834578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:05.834578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:05.834578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:05.834578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:06.101823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:06.102398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:06.102616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:06.103615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:06.104072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:06.104381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.105486 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.107382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.108147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:06.108736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:06.109546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.110581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:06.110581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:06.110581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.110581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:06.110581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.110581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:06.144794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:06.145302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:06.145421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:06.145566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:06.146133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:06.146358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.146970 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:06.147972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.148643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.149033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:06.149530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:06.150038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.150038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:06.150038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.150038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:06.150204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.150841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.151489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:06.154022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:06.154403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:06.154521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:06.154658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:06.155019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:06.155364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.156094 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:06.156978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.157418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.158143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:06.158567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:06.159030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.159030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:06.159030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.159030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.159563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:06.160101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:06.160629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.160629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:06.160629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.160629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:06.185025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:06.185570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:06.185728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:06.185911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:06.186408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:06.186907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:06.187232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.188145 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.189217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.189568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:06.189919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.189919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:06.190773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:06.191188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:06.191673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:06.191673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:06.191673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:10.785707 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:10.786652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:10.786652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:10.787622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:10.829813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:10.830098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:10.830831 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:10.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:10.832579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:10.833417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:10.834340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:10.835981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:10.835981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:10.835981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:10.835981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:10.835981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:10.835981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:11.102545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:11.103118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:11.103411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:11.104398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:11.104683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:11.104876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.105659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.107609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.108377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:11.109069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:11.109857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.110653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:11.110653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:11.110653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.110653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:11.110653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.110653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:11.145481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:11.145884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:11.145965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:11.146059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:11.146589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:11.146957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.147792 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:11.148530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.149084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.149368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:11.149732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:11.150130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.150524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.151064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:11.154454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:11.154736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:11.154910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:11.155081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:11.155568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:11.155956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.156630 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:11.159074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.159658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.160992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:11.161453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:11.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:11.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.162036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.162949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:11.163622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:11.164278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.164278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:11.164278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.164278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:11.185830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:11.186395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:11.186525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:11.186719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:11.187355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:11.187797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:11.188075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.188914 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.189925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:11.191180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:11.191905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:11.192819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:11.192819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:11.192819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:11.192819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:11.192819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:11.192819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:15.786277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:15.787264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:15.787264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:15.788418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:15.824337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:15.824592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:15.825319 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:15.826122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:15.826630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:15.827122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:15.827639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:15.828354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:15.828354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:15.828354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:15.828354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:15.828354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:15.828354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:16.103345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:16.103881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:16.104001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:16.104874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:16.105104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:16.105283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.105924 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.107230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.108025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:16.108875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.108875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:16.109023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:16.109919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.111169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:16.111169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.111169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:16.146131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:16.146574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:16.146751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:16.146985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:16.147457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:16.147724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.148521 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:16.149940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.150424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.151294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:16.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:16.152542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.152542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:16.152542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.152542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:16.153361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.153988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.154669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:16.154945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:16.155253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:16.155487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:16.156362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:16.156424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:16.156806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.157499 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:16.158073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.158743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:16.159104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:16.159490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.159490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:16.159490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.159490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.159635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:16.160223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:16.160780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.160780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:16.160780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.160780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:16.186498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:16.187152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:16.187345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:16.187648 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:16.188144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:16.188521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:16.188744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.189588 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.191216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.191649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:16.192047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.192047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:16.193363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:16.193938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:16.194484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:16.194484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:16.194484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:20.787137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:20.788042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:20.788042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:20.789092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:20.791481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=434 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:56:20.828934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:20.829129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:20.829751 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:20.830850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:20.831470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:20.832222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:20.833610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:20.835370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:20.835370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:20.835370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:20.835370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:20.835370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:20.835370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:21.103909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:21.104300 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:21.104492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:21.105149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:21.105376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:21.105581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.106237 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.107263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.107661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:21.108139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.108139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:21.108560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:21.109669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.110440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:21.110440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.110440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:21.146875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:21.147242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:21.147423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:21.147633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:21.148140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:21.148500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.149428 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:21.150577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.151297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.151894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:21.152587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:21.153480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.153480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:21.153480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.153480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:21.153803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.154675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:21.155802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:21.156501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:21.156501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:21.156988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.157691 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:21.158427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.158726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.159320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:21.159538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:21.159814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.159814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:21.159814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.159814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.160425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:21.160823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:21.161179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.161179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:21.161179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.161179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:21.187192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:21.187472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:21.187643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:21.187898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:21.188341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:21.188766 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:21.189099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.189963 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.191195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.191723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:21.192348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.192348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:21.192717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:21.194098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:21.194098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:21.194098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:25.787912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:25.788989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:25.788989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:25.790070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:25.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:25.834790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:25.835716 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:25.836587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:25.837246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:25.838181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:25.838918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:25.839560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:25.839560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:25.839560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:25.839560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:25.839560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:25.839560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:26.104566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:26.105245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:26.105365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:26.106296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:26.106709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:26.107053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.108162 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.109528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.110183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:26.110638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.110638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:26.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:26.111685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.112416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:26.112416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.112416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:26.147587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:26.147896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:26.148060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:26.148214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:26.148811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:26.149314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.150261 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:26.151147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.152029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.152593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:26.153367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:26.153916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.154323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.154905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:26.155621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:26.155887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:26.156032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:26.156205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:26.156563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:26.156923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.157420 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:26.158293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.158850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.159493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:26.160001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:26.160548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.160548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:26.160548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.160548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.161055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:26.161716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:26.162075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.162075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:26.162075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.162075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:26.187853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:26.188438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:26.188565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:26.188852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:26.189390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:26.189815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:26.189815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.190428 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.191422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:26.192315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.192315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:26.192746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:26.193270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:26.193869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:26.193869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:26.193869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:30.788734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:30.790570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:30.790903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:30.792916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:30.828496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:30.828892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:30.829870 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:30.831097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:30.832613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:30.833323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:30.834289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:30.836209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:30.836209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:30.836209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:30.836209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:30.836209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:30.836209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:31.105213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:31.106139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:31.106338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:31.107811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:31.108055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:31.108284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.109219 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.111317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.111966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:31.112505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.112505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:31.113145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:31.114857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.115330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:31.115330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.115330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:31.148190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:31.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:31.149077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:31.149299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:31.150530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:31.150779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.151589 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:31.153126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.153987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.154732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:31.156103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:31.157358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.158707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:31.160330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.160953 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:31.161483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:31.162180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:31.162827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:31.163416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:31.163970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.164940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:31.188450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:31.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:31.189285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:31.189529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:31.190161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:31.190527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:31.190800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.191568 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.192838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.193474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:31.194283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:31.195237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:31.196107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:31.196107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:31.196107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:31.196107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:31.196107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:31.196107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:35.790802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:35.791753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:35.791753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:35.792637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:35.830048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:35.830459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:35.831459 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:35.832213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:35.833068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:35.833706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:35.834531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:35.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:35.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:35.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:35.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:35.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:35.835486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:36.106145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:36.106504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:36.106693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:36.108053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:36.108503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:36.108899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.110080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.112132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.112720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:36.113394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.113394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:36.114277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:36.115148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.116021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:36.116021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.116021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:36.148839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:36.149222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:36.149302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:36.149357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:36.150119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:36.150460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.151282 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:36.152022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.152560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.152992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:36.153360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:36.153833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.153833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:36.153833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.153833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:36.155124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.155544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.155918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:36.158147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:36.158387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:36.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:36.158577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:36.158940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:36.159173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.159726 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:36.160307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.160698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.160980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:36.161298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:36.161643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.161643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:36.161643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.161643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.161750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:36.162341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:36.162877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.162877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:36.162877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.162877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:36.188956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:36.189346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:36.189536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:36.189813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:36.190310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:36.190555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:36.190702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.191390 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.193272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.193611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:36.193967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.193967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:36.195150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:36.195699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:36.196213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:36.196213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:36.196213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:40.791413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:40.792355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:40.792355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:40.793154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:40.860671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:40.861100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:40.862032 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.070 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:40.862876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:40.863487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:40.864069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:40.864755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:40.865682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:40.865682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:40.865682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:40.865682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:40.865682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:40.865682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:41.106610 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:41.108319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:41.108319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:41.108958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:41.109302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:41.109679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.110918 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.111862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.112370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:41.112862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.112862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:41.113378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:41.114311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.114815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:41.114815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.114815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:41.149382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:41.150053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:41.150294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:41.150594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:41.151448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:41.151906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.153019 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:41.154555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.155481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.157054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:41.159232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:41.160839 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.161268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:41.161983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.162554 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:41.165236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:41.165600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:41.166048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:41.166344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:41.166557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.166942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:41.189481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:41.189917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:41.190094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:41.190307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:41.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:41.191302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:41.191528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.192335 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.193331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.193861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:41.194476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.194476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:41.194729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:41.195234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:41.195724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:41.195724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:41.195724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:45.792310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:45.793663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:45.793917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:45.795217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:45.830781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:45.831282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:45.832358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:45.833157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:45.833812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:45.834488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:45.835291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:45.836405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:45.836405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:45.836405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:45.836405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:45.836405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:45.836405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:46.107433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:46.107937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:46.108179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:46.109355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:46.109684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:46.110018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.110904 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.112331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.112718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:46.113099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.113099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:46.114086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:46.114555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.115076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:46.115076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.115076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:46.149929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:46.150241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:46.150397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:46.150565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:46.151219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:46.151491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.152237 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:46.153561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.154404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.155308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:46.155954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:46.156643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.156643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:46.156643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.156643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:46.157618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.158384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:46.159459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:46.159884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:46.159884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:46.160052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:46.160456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:46.160796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.161358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:46.162210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.163463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:46.163813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:46.164197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.164197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:46.164197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.164197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.165075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:46.165618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:46.166121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.166121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:46.166121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.166121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:46.190097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:46.190522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:46.190788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:46.191103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:46.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:46.192060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:46.192292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.193138 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.194697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.195338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:46.195988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.195988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:46.197020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:46.197547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:46.198063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:46.198063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:46.198063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:50.794432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:50.795523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:50.795523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:50.796858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:50.830121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:50.830632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:50.831546 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:50.832532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:50.833338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:50.834259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:50.835285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:50.836399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:50.836399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:50.836399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:50.836399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:50.836399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:50.836399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:51.108121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:51.108710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:51.108947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:51.110010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:51.110281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:51.110487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.111207 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.112754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.113528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:51.114163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:51.114927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.115998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:51.115998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:51.115998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.115998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:51.115998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.115998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:51.150527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:51.151116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:51.151324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:51.151521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:51.152238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:51.152472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.153379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:51.155078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.156012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.156491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:51.157163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:51.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:51.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.157909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:51.158329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.159446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:51.159881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:51.160293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:51.160477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:51.160622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:51.161082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:51.161371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.161931 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:51.162663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.163310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.163745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:51.164217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:51.164576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.164576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:51.164576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.164576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.165172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:51.165859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:51.166567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.166567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:51.166567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.166567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:51.190669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:51.191266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:51.191452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:51.191743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:51.192363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:51.192641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:51.192828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.193531 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.195220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.195871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:51.196486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.196486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:51.196919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:51.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:51.198451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:51.198451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:51.198451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:55.795408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:55.796249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:55.796249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:55.796852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:55.832925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:55.833086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:55.833573 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:55.834648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:55.835330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:55.836110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:55.836857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:55.837828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:55.837828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:55.837828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:55.837828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:55.837828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:55.837828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:56.116550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:56.116550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:56.116550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:56.116550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:56.116550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:56.116550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:56.116908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:56.117312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.118055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.118655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.119175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:56.151177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:56.151624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:56.151978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:56.152276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:56:56.152869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:56.153215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.154092 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:56.155299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.155681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.156783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:56.157134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:56.157485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.157485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:56.157485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.157485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:56.158769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.159169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.159606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:56.160482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:56.161216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:56.161306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:56.161444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:56.161809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:56.162119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.162607 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:56:56.163285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.163833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.165043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:56.165583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:56.166191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:56.166827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.167642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:56:56.191241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:56:56.191779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:56:56.191984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:56:56.192273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:56:56.192684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:56.193102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:56:56.193329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.194515 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.196097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.196680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:56:56.197297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.197297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:56:56.197551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:56:56.198287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:56:56.199121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:56:56.199121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:56:56.199121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:00.795971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:00.796715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:00.796886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:00.798169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:00.837508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:00.837935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:00.838778 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:00.839549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:00.840211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:00.840937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:00.841666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:00.842656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:00.842656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:00.842656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:00.842656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:00.842656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:00.842656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:01.109612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:01.110239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:01.110431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:01.111619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:01.112018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:01.112278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.113186 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.114972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.115646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:01.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:01.117510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.118402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:01.118402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:01.118402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.118402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:01.118402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.118402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:01.152108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:01.152361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:01.152570 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:01.152798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:01.153636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:01.153739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.154931 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:01.156128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.157031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.157475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:01.158286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:01.159140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.159140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:01.159140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.159140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:01.159343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.161104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:01.162130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:01.162739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:01.162739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:01.162739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:01.164157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:01.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:01.166177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:01.166992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:01.167631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.168486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:01.191680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:01.192056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:01.192234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:01.192471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:01.193306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:01.193699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:01.193981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.194983 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.196161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.196713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:01.197247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.197247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:01.199992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:01.200864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:01.202426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:01.202426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:01.202426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:05.796640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:05.797987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:05.797987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:05.799167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:05.833849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:05.834222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:05.834972 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:05.836002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:05.837233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:05.838131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:05.838578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:05.839143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:05.839143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:05.839143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:05.839143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:05.839143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:05.839143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:06.110371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:06.111034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:06.111263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:06.112380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:06.112749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:06.113037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.114024 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.115562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.115946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:06.116271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.116271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:06.117550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:06.118121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.118590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:06.118590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.118590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:06.152534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:06.153079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:06.153191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:06.153437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:06.154943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:06.154943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.161677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.187062 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:06.188163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.189159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.189871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:06.190580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:06.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:06.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.191278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:06.191514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.192261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:06.193209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:06.193802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:06.193802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:06.194320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.195006 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:06.195968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.196703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.197234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:06.198000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:06.198330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.198330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:06.198330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.198330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.198562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:06.198997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:06.199395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:06.199670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:06.199977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:06.200184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.200779 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.201693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.202210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:06.202612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.202612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:06.202733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:06.203337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:06.203918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:06.203918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:06.203918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:10.797351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:10.798278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:10.798278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:10.799871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:10.834380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:10.834598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:10.834975 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:10.835592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:10.836309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:10.836991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:10.837845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:10.838979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:10.838979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:10.838979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:10.838979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:10.838979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:10.838979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:11.110983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:11.111515 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:11.111718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:11.112969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:11.113335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:11.113610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.114531 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.116127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.116734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:11.117343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.117343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:11.118059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:11.118829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.119628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:11.119628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.119628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:11.153167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:11.153663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:11.153816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:11.153974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:11.154664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:11.155026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.155972 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:11.156622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:11.157516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.158346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.159041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:11.159595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.160831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.161775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:11.220249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:11.220249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:11.222078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:11.222078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:11.222078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:11.222078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:11.222078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.222821 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:11.223987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.225640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:11.226316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:11.227051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.227051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:11.227051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.227051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.227386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:11.228197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:11.228735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:11.229025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:11.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.229465 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.230033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.230422 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:11.230812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:11.231250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:11.231747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:11.231747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:11.231747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:11.231747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:11.231747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:11.231747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:15.798325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:15.799385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:15.799474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:15.801090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:15.838329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:15.838588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:15.839170 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:15.840951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:15.841678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:15.842540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:15.843653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:15.844818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:15.844818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:15.844818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:15.844818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:15.844818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:15.844818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:16.111808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:16.112466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:16.112726 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:16.113855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:16.114268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:16.114648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.115568 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.117375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.118031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:16.118752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.118752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:16.119299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:16.120108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.120689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:16.120689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.120689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:16.153862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:16.154440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:16.154609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:16.154705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:16.155167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:16.155401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.156340 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:16.157367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.158315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.158828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:16.159392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:16.160012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.160448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.161125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:16.221023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:16.222095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:16.223603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:16.223603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:16.223603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:16.223603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.223907 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:16.225024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.225856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.226433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:16.227177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:16.227634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.227634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:16.227634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.227634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.227960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:16.228514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:16.229147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:16.229446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:16.229446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:16.229522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.229850 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.230783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.231190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:16.231623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.231623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:16.231850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:16.232294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:16.232796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:16.232796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:16.232796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:20.799653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:20.800554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:20.800554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:20.801523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:20.803261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:57:20.841907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:20.841907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:20.842803 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:20.843554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:20.844228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:20.844626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:20.845240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:20.845981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:20.845981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:20.845981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:20.845981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:20.845981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:20.845981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:21.112484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:21.113041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:21.113265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:21.114365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:21.114696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:21.114971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.116059 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.117662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.118311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:21.118985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:21.119659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.120636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:21.120636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:21.120636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.120636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:21.120636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.120636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:21.154440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:21.154979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:21.155156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:21.155359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:21.156286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:21.156517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.157553 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:21.159029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.159640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.160437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:21.161046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:21.161540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.161540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:21.161540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.161540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:21.161842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.162233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.162787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:21.222251 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:21.222848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:21.222973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:21.223164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:21.223733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:21.224078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.224837 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:21.226295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.226752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:21.268322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:21.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:21.270374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:21.271357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:21.272012 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.272481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:21.273435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:21.274129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:21.274590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:21.274996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:21.274996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:21.274996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:21.274996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:21.274996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:21.274996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:25.800110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:25.801088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:25.801088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:25.802238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:25.834652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:25.835034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:25.835893 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:25.836844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:25.837610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:25.838421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:25.839392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:25.840476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:25.840476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:25.840476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:25.840476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:25.840476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:25.840476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:26.113259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:26.113655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:26.113753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:26.114470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:26.114674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:26.114861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.115373 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.116317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.116899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:26.117470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:26.118173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.119337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:26.119337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:26.119337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.119337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:26.119337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.119337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:26.155476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:26.155476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:26.155680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:26.155680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:26.156274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:26.156905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.157704 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:26.158876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:26.159956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.160663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.161312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:26.161959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.162691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.163559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:26.223020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:26.223442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:26.223625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:26.223925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:26.224911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:26.224911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.225723 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:26.227009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.227730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.228427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:26.229045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:26.229680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.229680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:26.229680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.229680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.230186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:26.231043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:26.231942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:26.232375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:26.232562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:26.232726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.233212 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.234087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.234526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:26.234926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.234926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:26.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:26.235500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:26.236090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:26.236090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:26.236090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:30.800683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:30.801620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:30.801620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:30.802827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:30.839725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:30.840299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:30.841248 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:30.842088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:30.842553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:30.843047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:30.843493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:30.844884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:30.844884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:30.844884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:30.844884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:30.844884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:30.844884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:31.114389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:31.114968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:31.115183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:31.116166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:31.116545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:31.117080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.119024 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.121668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.122567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:31.123234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.123234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:31.124152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:31.124993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.125897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:31.125897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.125897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:31.158236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:31.159117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:31.159372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:31.159586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:31.160563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:31.160910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.161845 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:31.163851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.164213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.167104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:31.167630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:31.168237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.168237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:31.168237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.168237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:31.169349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.170042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.170854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:31.223672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:31.224335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:31.224426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:31.224657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:31.226269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:31.226908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.227858 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:31.229241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.229976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.230876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:31.231468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:31.232103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.232103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:31.232103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.232103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.232296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:31.233675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:31.234614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.235420 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.235806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.236215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:31.236608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.236608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:31.237200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:31.237700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:31.238295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:31.238295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:31.238295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:35.801528 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:35.802541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:35.802541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:35.803657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:35.836614 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:35.837132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:35.838123 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:35.839051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:35.839730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:35.841138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:35.841933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:35.842663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:35.842663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:35.842663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:35.842663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:35.842663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:35.842663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:36.115251 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:36.115794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:36.116255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:36.117281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:36.117549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:36.117772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.118700 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.120411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.121642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:36.122315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.122315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:36.122783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:36.123292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.123847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:36.123847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.123847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:36.159918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:36.159918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:36.159918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:36.160224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:36.161149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:36.161480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.162423 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:36.163672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:36.164677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.165508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.166065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:36.166553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.168884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.171182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:36.224308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:36.224996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:36.225214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:36.225469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:36.226385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:36.226814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.227693 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:36.229400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.230504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.231049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:36.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:36.233078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:36.233954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:36.234681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:36.235233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:36.235233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:36.235233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:36.235233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:36.235233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.235829 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.236651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.237170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:36.237645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:36.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:36.239198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:36.239198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:36.239198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:36.239198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:36.239198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:36.239198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:40.802144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:40.803269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:40.803269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:40.804545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:40.842587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:40.842958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:40.843705 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:40.844406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:40.844810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:40.845149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:40.845149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:40.845691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:40.846690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:40.847250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:40.847250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:40.847250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:41.115883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:41.116556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:41.116896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:41.117979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:41.118266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:41.118510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.119165 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.120680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.121082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:41.121408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.121408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:41.121779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:41.122188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.122664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:41.122664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.122664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:41.159752 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:41.160331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:41.160442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:41.160580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:41.161223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:41.161540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.162358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:41.163355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.163845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.164390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:41.164857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:41.165231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.165231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:41.165231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.165231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:41.165737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.166198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.166726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:41.225288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:41.226006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:41.226145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:41.226273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:41.226956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:41.227232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.228223 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:41.229162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.229576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.230159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:41.230416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:41.230733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.230733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:41.230733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.230733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.231806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:41.232203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:41.232635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.232635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:41.232635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.232635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:41.233933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:41.234285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:41.234408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:41.234585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:41.234904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:41.235085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:41.235193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.235628 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.236398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.236976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:41.237493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:41.238076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:41.238833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:41.238833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:41.238833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:41.238833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:41.238833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:41.238833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:45.802813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:45.803994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:45.804116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:45.806130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:45.848571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:45.849020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:45.850086 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:45.850999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:45.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:45.852574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:45.853578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:45.854789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:45.854789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:45.854789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:45.854789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:45.854789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:45.854789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:46.116752 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:46.117587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:46.117805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:46.119562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:46.119965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:46.120211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.121237 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.122390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.123055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:46.123957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:46.124812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.125874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:46.125874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:46.125874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.125874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:46.125874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.125874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:46.160463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:46.161011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:46.161123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:46.161265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:46.161857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:46.162988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.165013 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:46.167162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:46.167903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.168353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.168730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:46.169188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.169625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.170300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:46.226268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:46.226959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:46.227096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:46.227267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:46.227913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:46.228311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.229083 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:46.229972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.230712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.231211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:46.231745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:46.232397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:46.233134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.234222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:46.234802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:46.234802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:46.234960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:46.235053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:46.235506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:46.235907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:46.236153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.238078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.238551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:46.238906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.238906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:46.239383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:46.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:46.240368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:46.240368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:46.240368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:50.803821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:50.804641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:50.804641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:50.805310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:50.839852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:50.840045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:50.840784 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:50.841653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:50.842271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:50.842807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:50.842807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:50.842987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:50.843684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:50.844523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:50.844523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:50.844523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:51.117686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:51.118329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:51.118573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:51.119795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:51.120198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:51.120514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.121517 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.123192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.123869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:51.124650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.124650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:51.125459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:51.126443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.127410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:51.127410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.127410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:51.161220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:51.161669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:51.161925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:51.162153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:51.163339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:51.163711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.164726 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:51.166160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.167008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.167515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:51.168094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:51.168696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.168696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:51.168696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.168696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:51.169367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.170114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.170954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:51.227002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:51.227460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:51.227641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:51.227871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:51.228568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:51.229101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.229934 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:51.231289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.231987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.232807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:51.233380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:51.234023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.234023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:51.234023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.234023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.234692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:51.235473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:51.236015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:51.236295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:51.236499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:51.236745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.237548 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.238847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.239214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:51.239555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.239555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:51.240359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:51.241076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:51.241653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:51.241653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:51.241653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:55.804489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:55.805114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:55.805210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:55.806638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:55.838618 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:55.839034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:55.839904 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:55.841059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:55.841464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:55.841810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:55.841810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:55.842431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:55.842888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:55.843404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:55.843404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:55.843404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:56.118493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:56.118756 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:56.119076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:56.120171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:56.120447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:56.120628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.121436 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.123090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.123669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:56.124080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.124080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:56.124911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:56.125400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.125991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:56.125991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.125991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:56.161934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:56.162894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:56.163109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:56.163314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:57:56.164037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:56.164354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.165254 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:56.166882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.167327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.168523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:56.168917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:56.169287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.169287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:56.169287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.169287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:56.169645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.170107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.170593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:56.227822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:56.228435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:56.228598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:56.228879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:56.229668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:56.230137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.231067 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:57:56.232563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:56.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:56.234577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:56.235464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:57:56.235846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.236230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:57:56.236720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:57:56.236720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:57:56.236720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:57:56.236720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:57:56.236720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:56.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:57:56.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.238243 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.239092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.239488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:57:56.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.239874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:57:56.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:57:56.240594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:57:56.241013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:57:56.241013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:57:56.241013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:00.805466 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:00.806408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:00.806408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:00.806995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:00.840577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:00.841080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:00.841957 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:00.842692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:00.843356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:00.844047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:00.844799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:00.845710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:00.845710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:00.845710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:00.845710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:00.845710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:00.845710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:01.119512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:01.120249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:01.120501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:01.122088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:01.122400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:01.122579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.123486 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.125234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.125975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:01.126632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.126632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:01.127093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:01.128109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.129002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:01.129002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.129002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:01.162979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:01.163420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:01.163529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:01.163687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:01.164521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:01.165009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.166056 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:01.166885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.167416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.167971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:01.168468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:01.168964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.168964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:01.168964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.168964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:01.169435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.170123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.170611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:01.228496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:01.228900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:01.229108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:01.229322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:01.229934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:01.230414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.231213 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:01.232381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.233140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.233791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:01.234486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:01.235107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.235107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:01.235107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.235107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.235547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:01.236333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:01.237126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:01.237549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:01.237549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:01.237657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.238372 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.239629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.240041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:01.240426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.240426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:01.241589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:01.242089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:01.242596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:01.242596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:01.242596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:05.806496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:05.807437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:05.807437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:05.808202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:05.840807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:05.840973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:05.841410 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:05.842651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:05.843534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:05.844290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:05.844987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:05.845556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:05.845556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:05.845556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:05.845556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:05.845556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:05.845556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:06.120335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:06.120861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:06.120976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:06.121859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:06.122201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:06.122659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.123283 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.124367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.125263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:06.125864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:06.126492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.127318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:06.127318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:06.127318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.127318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:06.127318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.127318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:06.163514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:06.164388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:06.164497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:06.165070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:06.165902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:06.166290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.167446 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:06.168331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.168887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.169227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:06.169575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:06.169978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.169978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:06.169978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.169978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:06.170321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.170840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.171534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:06.229110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:06.229749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:06.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:06.230102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:06.230918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:06.231359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.232365 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:06.233516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.234064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.234468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:06.234966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:06.235353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:06.235792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:06.236397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:06.236809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:06.236853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:06.237015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:06.237458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:06.237952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:06.238202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.238916 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.239707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.240081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:06.240469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.240469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:06.240686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:06.241149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:06.241659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:06.241659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:06.241659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:10.807269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:10.808048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:10.808173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:10.809481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:10.847983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:10.848120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:10.848730 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:10.849644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:10.850157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:10.850550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:10.850550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:10.850739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:10.851264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:10.851879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:10.851879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:10.851879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:11.121095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:11.121629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:11.121848 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:11.122940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:11.123229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:11.123427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.124260 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.126099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.127204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:11.127911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:11.128412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.128951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:11.128951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:11.128951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.128951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:11.128951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.128951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:11.164169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:11.164786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:11.164954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:11.165216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:11.166009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:11.166436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.167539 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:11.168413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:11.169345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.170317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.170987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:11.171633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.172717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.173457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:11.229822 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:11.230406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:11.230587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:11.230788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:11.231659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:11.232199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.233185 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:11.234608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.235577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.236195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:11.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:11.237988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.237988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:11.237988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.237988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.238272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:11.239253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:11.240433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:11.240847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:11.240847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:11.240847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.241073 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.241898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.242277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:11.242635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.242635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:11.242855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:11.243333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:11.244010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:11.244010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:11.244010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:15.807949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:15.808630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:15.808876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:15.810412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:15.846244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:15.846913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:15.847848 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:15.848786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:15.849257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:15.849681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:15.849681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:15.850196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:15.850796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:15.851411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:15.851411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:15.851411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:16.121813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:16.122315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:16.122562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:16.123682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:16.123990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:16.124250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.125230 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.127492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.128508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:16.129399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.129399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:16.129555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:16.130408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.131132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:16.131132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.131132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:16.164925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:16.165372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:16.165534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:16.165706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:16.166368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:16.166560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.167254 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:16.168749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.170073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.170827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:16.171695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:16.172635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.172635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:16.172635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.172635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:16.173229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.174122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:16.230524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:16.231067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:16.231263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:16.231485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:16.232297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:16.232711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.233605 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:16.235166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.236053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.236721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:16.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:16.237469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.237469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:16.237469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.237469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:16.239223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:16.239754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:16.240123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:16.240272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:16.240466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.241081 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.242011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.242435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:16.242753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.242753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:16.243274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:16.243686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:16.244216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:16.244216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:16.244216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:20.808634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:20.809590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:20.809590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:20.810658 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:20.812437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:58:20.844789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:20.845047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:20.845636 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:20.846390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:20.846877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:20.847278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:20.847800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:20.848617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:20.848617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:20.848617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:20.848617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:20.848617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:20.848617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:21.122429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:21.122939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:21.123153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:21.124025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:21.124291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:21.124600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.125620 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.127333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.127943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:21.128542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.128542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:21.129246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:21.130046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.130909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:21.130909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.130909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:21.165645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:21.166156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:21.166348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:21.166610 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:21.167257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:21.167497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.168376 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:21.169679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.170605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.171268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:21.171928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:21.172709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.172709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:21.172709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.172709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:21.173265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.173741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.174343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:21.231116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:21.231634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:21.232046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:21.232046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:21.232574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:21.232849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.233434 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:21.234780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.235685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.236506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:21.237578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:21.238333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:21.238999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:21.240171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:21.241021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:21.241021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:21.241021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:21.241212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:21.241964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.242747 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.243602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:21.244415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.244415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:21.244690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:21.245174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:21.245687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:21.245687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:21.245687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:25.809264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:25.810190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:25.810190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:25.812066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:25.850055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:25.850239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:25.850636 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:25.851344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:25.851986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:25.852646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:25.853520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:25.854664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:25.854664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:25.854664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:25.854664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:25.854664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:25.854664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:26.123533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:26.124264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:26.124508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:26.126096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:26.126272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:26.126544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.127995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.129327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.130029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:26.130674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.130674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:26.131986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:26.132327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.132701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:26.132701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.132701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:26.166815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:26.167098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:26.167098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:26.167216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:26.167878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.170796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.171344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.171815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:26.172153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:26.172518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.172518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:26.172518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.172518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:26.174843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.175470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.175982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:26.231915 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:26.232321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:26.232521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:26.232729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:26.233502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:26.233957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.234849 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:26.236167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.236570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.237664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:26.238035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:26.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:26.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.238425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.239548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:26.240087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:26.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:26.241026 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:26.241390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:26.241636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.242410 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.243288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.243691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:26.244074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.244074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:26.244321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:26.244778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:26.245297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:26.245297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:26.245297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:30.810412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:30.811215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:30.811375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:30.812507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:30.848903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:30.849375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:30.850367 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:30.851130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:30.851775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:30.852842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:30.853973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:30.855244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:30.855244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:30.855244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:30.855244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:30.855244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:30.855244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:31.124400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:31.125102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:31.125102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:31.126203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:31.126597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:31.127019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.127996 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.129483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.129898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:31.130263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.130263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:31.131245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:31.131709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.132241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:31.132241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.132241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:31.167223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:31.167877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:31.168013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:31.168173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:31.168698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:31.168970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.169545 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:31.170427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:31.171540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.172614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.173028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:31.173388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.173851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:31.232574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:31.233125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:31.233256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:31.233465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:31.234148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:31.234571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.235592 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:31.236893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.237576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.239216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:31.240212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:31.241168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.241168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:31.241168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.241168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.242194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:31.242730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:31.243295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:31.243555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:31.243933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:31.244111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.244654 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.245805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.246450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:31.247106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.247106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:31.247452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:31.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:31.248478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:31.248478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:31.248478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:35.811728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:35.812574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:35.812758 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:35.814106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:35.849665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:35.849951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:35.850674 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:35.851744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:35.852055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:35.852342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:35.852342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:35.852885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:35.853255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:35.853622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:35.853622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:35.853622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:36.125428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:36.126110 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:36.126278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:36.127243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:36.127500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:36.127675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.128363 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.129539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.130241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:36.130738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:36.131267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.132013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:36.132013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:36.132013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.132013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:36.132013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.132013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:36.168320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:36.168795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:36.168916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:36.169059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:36.169743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:36.170379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.171474 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:36.172361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.172908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.173864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:36.174262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:36.174696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.174696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:36.174696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.174696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:36.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.177076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.177579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:36.233394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:36.233894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:36.234037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:36.234209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:36.235046 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:36.235632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.236800 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:36.238499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.239409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.240599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:36.241308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:36.242153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.242153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:36.242153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.242153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.242699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:36.243645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:36.244184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:36.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:36.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:36.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.244858 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.245717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.246041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:36.246378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.246378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:36.246774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:36.247205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:36.247721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:36.247721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:36.247721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:40.812473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:40.814815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:40.814894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:40.815520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:40.850450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:40.850704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:40.851296 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:40.852628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:40.853001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:40.853332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:40.853332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:40.854934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:40.855370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:40.855844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:40.855844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:40.855844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:41.126139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:41.126612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:41.126725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:41.127527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:41.127730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:41.127864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.128511 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.129602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.129933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:41.130223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.130223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:41.130571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:41.130933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.131297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:41.131297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.131297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:41.170690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:41.171200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:41.171404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:41.171603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:41.172233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:41.172403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.173092 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:41.174147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.174552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.175249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:41.175584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:41.175867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.175867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:41.175867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.175867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:41.176981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.177337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.177716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:41.236837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:41.236837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:41.236837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:41.236837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:41.236837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:41.236837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.237294 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:41.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.238281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.240002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:41.240297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:41.240601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.240601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:41.240601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.240601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.243564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:41.244042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:41.244409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.244409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:41.244409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.244409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:41.244409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:41.244875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:41.245641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:41.245971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:41.246779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:41.247188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:41.247351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.247972 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.249370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.249710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:41.249984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:41.250285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:41.250754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:41.250754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:41.250754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:41.250754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:41.250754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:41.250754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:45.814927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:45.815943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:45.815943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:45.816493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:45.849477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:45.849798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:45.852668 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:45.853494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:45.854342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:45.855755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:45.857103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:45.857728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:45.857728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:45.857728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:45.857728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:45.857728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:45.857728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:46.126692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:46.127239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:46.127531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:46.128731 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:46.129004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:46.129166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.129961 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.131818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.132684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:46.133272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.133272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:46.134229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:46.135029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.135531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:46.135531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.135531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:46.171460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:46.172304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:46.172861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:46.173261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:46.173992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:46.174205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.174995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:46.176639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.177183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.178733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:46.179550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:46.179939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.179939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:46.179939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.179939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:46.180718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.181153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.181515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:46.234523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:46.235089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:46.235286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:46.235530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:46.236202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:46.236550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.237224 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:46.238686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.239464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.241229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:46.241639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:46.242044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.242044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:46.242044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.242044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.243909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:46.244524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:46.245121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.245121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:46.245121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.245121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:46.245121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:46.245336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:46.245421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:46.245532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:46.246089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:46.246349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:46.246501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.247061 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.247644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.247962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:46.248228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.248228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:46.248833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:46.249272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:46.249682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:46.249682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:46.249682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:50.815568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:50.816263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:50.816263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:50.817424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:50.851248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:50.851467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:50.852247 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:50.853983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:50.854318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:50.854644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:50.854644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:50.856101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:50.856504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:50.856924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:50.856924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:50.856924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:51.127353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:51.127704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:51.127846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:51.129642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:51.130182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:51.130538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.131767 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.133491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.133990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:51.134326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.134326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:51.135478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:51.136277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.136804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:51.136804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.136804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:51.172064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:51.172545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:51.172795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:51.172914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:51.173815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:51.174242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.175135 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:51.176933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.177731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.178852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:51.179371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:51.179784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.179784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:51.179784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.179784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:51.180997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.181499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.182173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:51.235077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:51.235572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:51.235659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:51.235920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:51.236432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:51.236727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.237229 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:51.238972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.239672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.240591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:51.241267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:51.241894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.241894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:51.241894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.241894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.242533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:51.243248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:51.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:51.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.244015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:51.245510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:51.245912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:51.246274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:51.246439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:51.247103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:51.247492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:51.247746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.248528 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.249357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.249723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:51.250273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.250273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:51.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:51.250956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:51.252000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:51.252000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:51.252000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:55.816166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:55.817060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:55.817340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:55.818786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:55.852425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:55.852870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:55.853629 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:55.854383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:55.855062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:55.855754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:55.856564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:55.858280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:55.858280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:55.858280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:55.858280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:55.858280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:55.858280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:56.128028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:56.128674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:56.128961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:56.130069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:56.130501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:56.130872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.131922 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.133483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.134086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:56.134688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.134688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:56.135942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:56.136468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.137108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:56.137108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.137108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:56.172708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:56.173294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:56.173571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:56.173837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:58:56.174484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:56.174906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.175747 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:56.177101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.177996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.178603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:56.179277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:56.179967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.179967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:56.179967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.179967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:56.180456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.181157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.181908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:56.235703 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:56.236224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:56.236430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:56.236630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:56.237358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:56.237789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.238659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:58:56.239974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.240633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.241535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:56.242199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:56.242993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.242993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:56.242993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.242993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.243619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:56.244381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:56.244944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.244944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:56.244944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.244944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:58:56.245930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:58:56.246230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:58:56.246361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:58:56.246509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:58:56.246960 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:56.247211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:58:56.247374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.247965 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.248869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.249217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:58:56.249589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.249589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:58:56.249922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:58:56.250352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:58:56.250827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:58:56.250827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:58:56.250827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:00.816948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:00.817601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:00.817601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:00.819079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:00.854456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:00.856148 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:00.856709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:00.857327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:00.857919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:00.858696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:00.859524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:00.859524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:00.859524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:00.859524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:00.859524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:00.859524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:01.128936 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:01.129563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:01.129718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:01.131055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:01.131416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:01.131667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.132583 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.134163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.135261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:01.135824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.135824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:01.135898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:01.136388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.136992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:01.136992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.136992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:01.173353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:01.173787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:01.174050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:01.174361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:01.174885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:01.175194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.176070 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:01.177304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.178158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.178677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:01.179352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:01.180174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.180174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:01.180174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.180174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:01.180384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.181233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.182125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:01.236440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:01.236975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:01.237097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:01.237255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:01.237863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:01.238297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.239158 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:01.240084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:01.241223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:01.242229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:01.243229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:01.244041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.245289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:01.246627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:01.247283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:01.247283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:01.247283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:01.247521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:01.247905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:01.248092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.248884 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.249866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.250332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:01.250950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:01.251490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:01.252088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:01.252088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:01.252088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:01.252088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:01.252088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:01.252088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:05.817705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:05.818406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:05.818406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:05.819048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:05.853561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:05.854001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:05.854875 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:05.855668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:05.856161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:05.856501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:05.856919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:05.857398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:05.857398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:05.857398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:05.857398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:05.857398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:05.857398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:06.129706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:06.130096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:06.130096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:06.131103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:06.131593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:06.131812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.132847 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.133685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.134109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:06.134633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:06.135156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.135798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:06.135798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:06.135798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.135798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:06.135798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.135798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:06.173934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:06.174462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:06.174622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:06.174964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:06.175226 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:06.175457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.176436 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:06.177287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.178210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.178712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:06.179246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:06.179869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.180502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.181506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:06.236988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:06.237566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:06.237745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:06.237951 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:06.239028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:06.239286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.239915 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:06.241031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.241686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.242714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:06.243273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:06.243975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.243975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:06.243975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.243975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.245590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:06.246238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:06.246825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.246825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:06.246825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.246825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:06.246825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:06.248239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:06.248350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:06.248524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:06.248884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:06.249116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:06.249329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.250059 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.250920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.251326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:06.251861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:06.252390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:06.252965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:06.252965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:06.252965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:06.252965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:06.252965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:06.252965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:10.818267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:10.819047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:10.819047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:10.820572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:10.853752 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:10.854016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:10.854319 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:10.854986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:10.855935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:10.857404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:10.858407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:10.859966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:10.859966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:10.859966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:10.859966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:10.859966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:10.859966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:11.133527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:11.133527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:11.133527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:11.133527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:11.133527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:11.133527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:11.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:11.135650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.136650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.138124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.139724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:11.174593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:11.174888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:11.175059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:11.175259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:11.175652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:11.175926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.176495 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:11.177500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.178249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.179024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:11.179594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:11.180200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.180200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:11.180200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.180200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:11.180533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.181050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.181567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:11.237995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:11.238496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:11.238697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:11.239344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:11.239834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:11.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.241516 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:11.242970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:11.244053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:11.244885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:11.245884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:11.246687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.247385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:11.248187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:11.248590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:11.248756 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:11.248961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:11.249273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:11.249496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:11.249638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.250194 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.251094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:11.251735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.251735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:11.252162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:11.252585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:11.253151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:11.253151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:11.253151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:15.819006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:15.819858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:15.819858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:15.820459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:15.856631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:15.857669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:15.858399 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:15.859233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:15.859580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:15.859944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:15.859944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:15.860365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:15.860828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:15.861306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:15.861306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:15.861306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:16.130860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:16.131257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:16.131450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:16.132849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:16.133241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:16.133482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.134501 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.136182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.136885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:16.137544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.137544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:16.138473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:16.139337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.140268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:16.140268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.140268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:16.175138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:16.175577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:16.175691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:16.175823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:16.176525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:16.176984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.177801 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:16.178483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:16.179463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.180614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.181357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:16.182160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.183037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.183816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:16.238559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:16.239201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:16.239479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:16.239721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:16.240223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:16.240550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.241352 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:16.242650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.243397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.244198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:16.244860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:16.245528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.245528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:16.245528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.245528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.246380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:16.247405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:16.248449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.248449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:16.248449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.248449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:16.248906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:16.249012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:16.249155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:16.249304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:16.249580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:16.249828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:16.249974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.250429 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.251250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.251492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:16.251743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.251743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:16.252302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:16.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:16.253077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:16.253077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:16.253077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:20.819853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:20.820626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:20.820626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:20.822013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:20.824996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 01:59:20.856671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:20.857230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:20.858215 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:20.859025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:20.859632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:20.860159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:20.860756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:20.861707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:20.861707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:20.861707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:20.861707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:20.861707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:20.861707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:21.131362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:21.132112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:21.132394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:21.133727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:21.134204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:21.134486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.135558 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.136812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.137491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:21.137828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.137828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:21.137891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:21.138317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.138833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:21.138833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.138833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:21.175921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:21.176588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:21.176847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:21.177075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:21.177624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:21.177910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.178601 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:21.180169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.180858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.181978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:21.182505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:21.183084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.183084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:21.183084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.183084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:21.183946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.184395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:21.239401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:21.240059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:21.240253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:21.240472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:21.241292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:21.241619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.242385 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:21.243725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.244509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.246500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:21.247995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:21.249293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:21.250104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:21.251862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:21.252647 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.253094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:21.253591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:21.254072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:21.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:21.255372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:21.255372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:21.255372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:21.255372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:21.255372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:21.255372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:25.820476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:25.821885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:25.821885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:25.822860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:25.862031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:25.862476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:25.863711 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:25.864844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:25.865458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:25.865857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:25.865857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:25.866372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:25.867836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:25.869307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:25.869307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:25.869307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:26.131937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:26.133227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:26.133227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:26.133696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:26.134075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:26.134319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.136038 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.136633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.137275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:26.138041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:26.138856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.139978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:26.139978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:26.139978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.139978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:26.139978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.139978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:26.176527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:26.176833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:26.176939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:26.177097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:26.177725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:26.178063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.178737 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:26.179646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:26.180535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.181249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.181933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:26.182467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.183349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.184165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:26.240097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:26.240440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:26.240552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:26.240703 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:26.241353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:26.241871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.242754 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:26.243663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.244426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.244861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:26.245431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:26.246088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:26.246906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.248013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:26.250222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:26.250543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:26.250626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:26.250825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:26.251186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:26.251455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:26.251615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.252187 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.253036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.253388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:26.253752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.253752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:26.254065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:26.254494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:26.254988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:26.254988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:26.254988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:30.821044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:30.821681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:30.821681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:30.822558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:30.857538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:30.859314 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:30.859877 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:30.860568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:30.861288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:30.862037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:30.862890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:30.863891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:30.863891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:30.863891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:30.863891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:30.863891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:30.863891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:31.132874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:31.133161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:31.133161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:31.137426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:31.137426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:31.137426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.137699 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.138985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.139459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:31.139835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.139835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:31.140245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:31.140920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.141409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:31.141409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.141409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:31.177148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:31.177752 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:31.177975 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:31.178181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:31.179905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:31.179905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.180163 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:31.181331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.182305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.182829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:31.183551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:31.184355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.185321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.186822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:31.240665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:31.241063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:31.241255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:31.241459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:31.242898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:31.242898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.243128 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:31.244278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.244609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.245689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:31.245963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:31.246227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.246227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:31.246227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.246227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.247533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:31.248233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:31.248914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.248914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:31.248914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.248914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:31.250605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:31.250872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:31.250969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:31.251174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:31.251511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:31.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:31.252016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.253285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.253821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:31.254274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:31.254835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:31.255481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:31.255481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:31.255481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:31.255481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:31.255481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:31.255481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:35.821873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:35.822667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:35.822667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:35.823364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:35.856922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:35.857421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:35.858560 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:35.859349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:35.859656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:35.859985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:35.859985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:35.860118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:35.860552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:35.861099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:35.861099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:35.861099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:36.133708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:36.134115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:36.134278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:36.135151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:36.135353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:36.135483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.136066 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.137502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.138450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:36.139156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:36.140029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.141011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:36.141011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:36.141011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.141011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:36.141011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.141011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:36.178799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:36.179458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:36.179741 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:36.180146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:36.180637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:36.180835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.183080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:36.183643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:36.184885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.185655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.186442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:36.187384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.188294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.189128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:36.241314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:36.241886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:36.242115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:36.242465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:36.243078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:36.243490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.244412 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:36.245967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.246628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.247821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:36.248378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:36.248966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.248966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:36.248966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.248966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.250932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:36.251613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:36.252454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:36.252734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:36.253139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:36.253269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.253626 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.254290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.254650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:36.255025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.255025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:36.255276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:36.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:36.256315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:36.256315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:36.256315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:40.822292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:40.823294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:40.823294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:40.824381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:40.857254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:40.857486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:40.858007 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:40.858904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:40.859592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:40.860346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:40.861248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:40.864036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:40.864036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:40.864036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:40.864036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:40.864036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:40.864036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:41.134387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:41.134855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:41.135031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:41.135636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:41.135821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:41.135964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.136577 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.137621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.138731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:41.139857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:41.140483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.141352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:41.141352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:41.141352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.141352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:41.141352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.141352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:41.179370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:41.179864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:41.179986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:41.180171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:41.181006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:41.181316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.182164 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:41.183133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.183623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.184077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:41.184599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:41.185161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.185161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:41.185161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.185161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:41.185402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.187286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.190053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:41.242144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:41.242616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:41.242728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:41.242933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:41.243904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:41.243904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.245979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.246933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.247654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:41.248193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:41.248591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.248591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:41.248591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.248591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.248829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:41.249350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:41.249985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.249985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:41.249985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.249985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:41.251783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:41.252057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:41.252176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:41.252338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:41.253006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:41.253272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:41.253435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.254081 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.254939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.255447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:41.255893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.255893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:41.255976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:41.256543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:41.257213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:41.257213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:41.257213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:45.823032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:45.823692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:45.823692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:45.824333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:45.853032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:45.853432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:45.854688 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:45.855870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:45.856306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:45.856736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:45.856736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:45.857302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:45.857898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:45.858564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:45.858564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:45.858564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:46.135211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:46.135629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:46.135790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:46.136755 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:46.137216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:46.137445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.138253 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.139029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.139549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:46.140042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:46.141506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.142297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:46.142297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:46.142297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.142297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:46.142297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.142297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:46.179914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:46.180451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:46.180609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:46.180856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:46.181611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:46.181991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.182988 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:46.184277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.185497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.185946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:46.186277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:46.186689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.186689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:46.186689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.186689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:46.187783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.188299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.188849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:46.242808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:46.243527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:46.243700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:46.244000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:46.244994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:46.245395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.246439 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:46.247722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.248712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.249322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:46.250158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:46.251076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:46.251560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:46.252311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:46.252661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:46.252661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:46.252661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:46.253177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:46.253540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:46.254003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.254739 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.255652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.256166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:46.256528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.256528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:46.256982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:46.257501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:46.258231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:46.258231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:46.258231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:50.823684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:50.824290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:50.824290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:50.825066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:50.860704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:50.861099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:50.861981 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:50.862835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:50.863153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:50.863457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:50.863457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:50.863701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:50.864077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:50.864465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:50.864465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:50.864465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:51.135884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:51.136444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:51.136498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:51.137440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:51.137679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:51.137874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.138820 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.140236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.140514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:51.140825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.140825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:51.141769 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:51.142103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.142459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:51.142459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.142459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:51.180342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:51.180958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:51.181119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:51.181269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:51.181966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:51.182278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.183112 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:51.184106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.184606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.185087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:51.185462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:51.185851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.185851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:51.185851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.185851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:51.186404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.186920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.187513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:51.243618 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:51.244099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:51.244231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:51.244376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:51.245091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:51.245716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.246962 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:51.247910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:51.248854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:51.249578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.250224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.251535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:51.252535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:51.254659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:51.255103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:51.255103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:51.255212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.255733 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.256632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.256945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:51.257213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.257213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:51.258166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:51.258642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:51.259028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:51.259028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:51.259028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:55.824583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:55.825307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:55.825307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:55.826124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:55.862167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:55.862568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:55.863442 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:55.864194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:55.864682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:55.865174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:55.865890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:55.866538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:55.866538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:55.866538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:55.866538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:55.866538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:55.866538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:56.137084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:56.137317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:56.137386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:56.138818 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:56.139473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:56.139473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.140157 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.142543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.143002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:56.143356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.143356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:56.144242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:56.144785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.145553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:56.145553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.145553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:56.180986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:56.181424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:56.181661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:56.181949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 01:59:56.182512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:56.182784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.184118 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:56.185735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.186687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.187366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:56.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:56.189551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.190913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.192238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:56.244326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:56.244931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:56.245152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:56.245347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:56.246192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:56.246642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.247680 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 01:59:56.249193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.250138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.251038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:56.251660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:56.252395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.252395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:56.252395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.252395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.253386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:56.254060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:56.254632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.254632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:56.254632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.254632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 01:59:56.254632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 01:59:56.254632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 01:59:56.255000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 01:59:56.255195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 01:59:56.255651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:56.255939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 01:59:56.256117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.256644 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.257327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 01:59:56.258193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.258193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 01:59:56.258257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 01:59:56.258896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 01:59:56.259442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 01:59:56.259442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 01:59:56.259442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:00.825233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:00.825909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:00.826077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:00.827445 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:00.857287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:00.857543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:00.858194 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:00.858976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:00.859915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:00.860999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:00.861813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:00.862700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:00.862700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:00.862700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:00.862700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:00.862700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:00.862700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:01.137163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:01.137653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:01.137862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:01.138844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:01.139248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:01.139560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.140522 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.142188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.142836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:01.143399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.143399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:01.143726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:01.144541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.145530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:01.145530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.145530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:01.181513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:01.182211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:01.182423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:01.182637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:01.183241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:01.183534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.184364 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:01.185304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:01.186439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.187404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.188442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:01.189152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.189578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.190045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:01.244921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:01.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:01.245640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:01.245830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:01.246579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:01.247162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.248141 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:01.249262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.249983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.251650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:01.252466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:01.253358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.253358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:01.253358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.253358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.254135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:01.255301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:01.256091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:01.256449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:01.256449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:01.256449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.256870 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.258134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.258567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:01.258933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.258933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:01.259623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:01.260127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:01.260615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:01.260615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:01.260615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:05.825794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:05.826603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:05.826917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:05.828593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:05.864252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:05.864638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:05.865579 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:05.866371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:05.867286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:05.867996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:05.868794 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:05.869847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:05.869847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:05.869847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:05.869847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:05.869847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:05.869847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:06.137968 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:06.138670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:06.138992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:06.140181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:06.140495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:06.140742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.141698 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.144034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.144717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:06.145235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.145235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:06.145611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:06.146234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.146819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:06.146819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.146819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:06.182120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:06.182720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:06.182964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:06.183199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:06.183842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:06.184160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.185096 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:06.186347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.187140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.188459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:06.189081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:06.189675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.189675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:06.189675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.189675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:06.190284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.191024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.192019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:06.245792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:06.246421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:06.246727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:06.247024 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:06.247532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:06.247868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.248546 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:06.250160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.251246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.252476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:06.253061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:06.253677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.253677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:06.253677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.253677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.254196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:06.254890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:06.255553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.255553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:06.255553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.255553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:06.255553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:06.255553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:06.255918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:06.255984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:06.256361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:06.256668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:06.256908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.257525 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.258529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.259139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:06.259803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.259803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:06.260009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:06.260450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:06.260885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:06.260885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:06.260885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:10.826480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:10.827007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:10.827072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:10.827895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:10.863851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:10.863851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:10.864291 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:10.865070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:10.865701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:10.866601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:10.867454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:10.868476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:10.868476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:10.868476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:10.868476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:10.868476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:10.868476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:11.138586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:11.139059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:11.139237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:11.140215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:11.140538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:11.140775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.141783 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.143281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.143643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:11.144056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.144056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:11.145094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:11.145573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.146119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:11.146119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.146119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:11.182796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:11.183276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:11.183497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:11.183702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:11.184189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:11.184402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.184981 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:11.186487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.187296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.188385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:11.189027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:11.189810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.189810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:11.189810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.189810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:11.190333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.191054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.191872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:11.246399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:11.246983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:11.247202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:11.247479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:11.248126 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:11.248447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.249279 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:11.251022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.251831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.252283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:11.252877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:11.253573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:11.254379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:11.255777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:11.256136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:11.256136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:11.256292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:11.256613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:11.256885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:11.257034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.257562 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.258618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.259010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:11.259369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.259369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:11.259784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:11.260252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:11.260811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:11.260811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:11.260811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:15.827179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:15.828098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:15.828098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:15.828643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:15.881611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:15.882099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:15.882588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:15.883059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:15.883687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:16.139248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:16.139630 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:16.140299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:16.141366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:16.141699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:16.141893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.142993 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.144736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.145204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:16.145622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.145622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:16.146896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:16.147998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.149031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:16.149031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.149031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:16.183441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:16.184036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:16.184234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:16.184470 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:16.185140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:16.185472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.186372 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:16.187927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.188899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.193832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:16.194337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:16.194684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.195099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.195595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:16.247009 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:16.247542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:16.247742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:16.248205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:16.248462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:16.248784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.249878 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:16.250852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.251476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.252346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:16.252938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:16.253478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.253478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:16.253478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.253478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.254715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:16.255692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:16.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:16.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:16.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:16.256532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:16.256935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:16.256935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:16.257146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:16.257361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:16.257537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.258076 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.258865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.259485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:16.260086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:16.260836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:16.261925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:16.261925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:16.261925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:16.261925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:16.261925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:16.261925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:20.827718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:20.828809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:20.828809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:20.829699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:20.831804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:00:20.873214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:20.873405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:20.874078 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:20.875223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:20.875828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:20.876369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:20.877018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:20.877735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:20.877735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:20.877735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:20.877735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:20.877735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:20.877735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:21.139829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:21.140130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:21.140321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:21.141339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:21.141638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:21.141902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.142502 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.144141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.144926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:21.146585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:21.147355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.149505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:21.149505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:21.149505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.149505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:21.149505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.149505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:21.184417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:21.185206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:21.185328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:21.185510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:21.186468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:21.186916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.188222 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:21.189638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.190448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.191425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:21.193037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:21.194041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.194659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.195639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:21.247920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:21.248724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:21.248964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:21.249187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:21.250240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:21.250803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.252078 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:21.253695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.255049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.255969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:21.256460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:21.256829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:21.257198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:21.257660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:21.258022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:21.258022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:21.258022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:21.258121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:21.258739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.258972 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.259860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.261081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:21.261967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:21.262625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:21.263153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:21.263153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:21.263153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:21.263153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:21.263153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:21.263153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:25.828671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:25.829912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:25.829912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:25.830151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:25.868081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:25.868708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:25.870105 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:25.871338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:25.871776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:25.872191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:25.872191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:25.872453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:25.873009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:25.873578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:25.873578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:25.873578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:26.140830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:26.141292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:26.141650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:26.143271 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:26.143677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:26.143986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.145059 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.146524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.147208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:26.147790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:26.148463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.149287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:26.149287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:26.149287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.149287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:26.149287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.149287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:26.185575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:26.186205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:26.186413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:26.186627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:26.187483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:26.187875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.188898 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:26.190481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.191294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.191935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:26.192560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:26.193108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.193108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:26.193108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.193108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:26.193594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.194052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.194503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:26.248828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:26.249369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:26.249573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:26.250000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:26.250880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:26.251359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.252498 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:26.254359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.254891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.256366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:26.256796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:26.257194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.257194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:26.257194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.257194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.258538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:26.259266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:26.259935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:26.260499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:26.260733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:26.261016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.261804 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.262466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.262916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:26.263509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:26.264107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:26.264640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:26.264640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:26.264640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:26.264640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:26.264640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:26.264640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:30.299706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:30.300670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:30.300670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:30.310021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 02:00:30.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 02:00:30.310311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:00:30.339406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.339862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.340631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.341086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.342124 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.343304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.343995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.344823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.344823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.344823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.344823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:00:30.374228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.374619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.375209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.375383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.376378 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.377625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.378359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.379208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.379208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.379208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.379208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:00:30.410673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.410981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.411330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.411462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.412159 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.413003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.413343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.413891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.413891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.413891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.413891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:00:30.444076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.444432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.444981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.445174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.445959 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.446958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.447582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.448516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.448516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.448516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.448516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:00:30.522871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.523262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.523672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.523949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.524244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.525207 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.525488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 02:00:30.527131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.527530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.527940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:00:30.644172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.644434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.644786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.645038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.645775 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.646669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.647119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.647535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.647535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.647535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.647535 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.346 seconds
May 05 02:00:30.829171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:30.829868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:30.830227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:30.831489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:30.863296 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:30.863685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.864639 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.865442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.866298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:30.866619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:30.867095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:30.867622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:30.867622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:30.867622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:30.867622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:30.867622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:30.867622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:31.141459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:31.142118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:31.142480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:31.143268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:31.143602 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:31.143874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.144856 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.146590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.147181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:31.147689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.147689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:31.148770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:31.149225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.149721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:31.149721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.149721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:31.186165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:31.186612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:31.186842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:31.187041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:31.187563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:31.187906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.188724 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:31.190008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.190820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.191908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:31.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:31.193844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.194468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.195562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:31.249507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:31.250124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:31.250396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:31.250667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:31.251237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:31.251635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.252659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:31.254149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.254862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.255702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:31.256277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:31.256942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.256942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:31.256942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.256942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.257551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:31.258352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:31.259123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.259123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:31.259123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.259123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:31.259499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:31.259926 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:31.260160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:31.260553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:31.260974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:31.261380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:31.261605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.262403 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.263177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.263692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:31.264024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.264024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:31.264116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:31.264572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:31.265035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:31.265035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:31.265035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:35.829976 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:35.831607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:35.831858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:35.833171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:35.870983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:35.871390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:35.872316 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:35.873137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:35.873865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:35.874536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:35.875433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:35.876611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:35.876611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:35.876611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:35.876611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:35.876611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:35.876611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:36.142148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:36.142632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:36.142930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:36.143785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:36.143990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:36.144134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.144946 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.146557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.147175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:36.147666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.147666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:36.148848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:36.149475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.150270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:36.150270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.150270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:36.188250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:36.188668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:36.188886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:36.189084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:36.189720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:36.189996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.190684 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:36.192248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.192676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.194446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:36.194807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:36.195192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.195192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:36.195192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.195192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:36.196141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.196560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.197000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:36.250100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:36.250428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:36.250551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:36.250689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:36.251442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:36.251890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.253081 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:36.254027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.254351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.255207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:36.255463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:36.255748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.255748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:36.255748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.255748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.257440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:36.257812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:36.258153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.258153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:36.258153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.258153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:36.261808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:36.261808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:36.261808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:36.261808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:36.261808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:36.262190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:36.262359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.263164 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.266169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.266598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:36.267577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:36.267961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:36.268349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:36.268349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:36.268349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:36.268349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:36.268349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:36.268349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:40.831653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:40.832472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:40.832472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:40.833290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:40.865304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:40.865494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:40.866506 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:40.868992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:40.869471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:40.869746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:40.869746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:40.870941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:40.871688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:40.872613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:40.872613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:40.872613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:41.142803 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:41.143324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:41.143519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:41.144525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:41.144896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:41.145132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.146061 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.148119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.148803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:41.149449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.149449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:41.149997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:41.150843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.151782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:41.151782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.151782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:41.188809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:41.189281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:41.189408 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:41.189557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:41.190187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:41.190542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.191460 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:41.193040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.193471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.194498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:41.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:41.195336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.195336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:41.195336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.195336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:41.196364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.196798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.197268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:41.250721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:41.251334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:41.251541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:41.251786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:41.252430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:41.252850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.253642 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:41.255206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.255983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.257010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:41.257623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:41.258620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.258620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:41.258620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.258620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.258919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:41.259912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:41.261071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.261071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:41.261071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.261071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:41.261071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:41.261351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:41.261456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:41.261621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:41.261962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:41.262162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:41.262349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.262823 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.263666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:41.264720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:41.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:41.265667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:41.265667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:41.265667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:41.265667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:41.265667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:41.265667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:45.832403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:45.833356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:45.833356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:45.834263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:45.866220 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:45.866450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:45.867029 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:45.867935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:45.868454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:45.869062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:45.869558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:45.870195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:45.870195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:45.870195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:45.870195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:45.870195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:45.870195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:46.143462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:46.143688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:46.143817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:46.144645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:46.144849 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:46.144968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.145457 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.146372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.146749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:46.147147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.147147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:46.147444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:46.148032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.148635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:46.148635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.148635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:46.189464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:46.190101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:46.190101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:46.190101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:46.190612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:46.190734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.191217 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:46.192027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.192706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.193072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:46.193604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:46.194116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.194551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.194998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:46.251283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:46.251825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:46.251825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:46.252020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:46.252601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:46.254026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.254722 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:46.255596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.255935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.256585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:46.257022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:46.257390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.257390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:46.257390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.257390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.257749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:46.258271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:46.258631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.258631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:46.258631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.258631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:46.261383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:46.261708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:46.262041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:46.262041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:46.262443 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:46.262649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:46.262771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.263459 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.264319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.264704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:46.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.265081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:46.265378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:46.266123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:46.266666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:46.266666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:46.266666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:50.833288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:50.834040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:50.834040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:50.834452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:50.871207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:50.871610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:50.872448 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:50.873238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:50.873802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:50.874343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:50.875058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:50.875827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:50.875827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:50.875827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:50.875827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:50.875827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:50.875827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:51.144025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:51.144363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:51.144455 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:51.145279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:51.145517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:51.145665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.146799 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.148871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.149197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:51.149528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.149528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:51.150059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:51.150421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.150942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:51.150942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.150942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:51.189987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:51.190247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:51.190394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:51.190536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:51.191157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:51.191400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.193055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:51.194003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.194363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.194972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:51.195321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:51.195619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.195619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:51.195619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.195619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:51.196351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.196901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.197567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:51.254101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:51.254384 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:51.254435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:51.254524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:51.255144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:51.255435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.255994 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:51.256665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.257020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.257293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:51.257810 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:51.258333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:51.258952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.259503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:51.262206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:51.262406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:51.262495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:51.262635 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:51.263142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:51.263496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:51.263650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.264302 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.265130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.265423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:51.265697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.265697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:51.265987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:51.266347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:51.266736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:51.266736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:51.266736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:55.833973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:55.834491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:55.834491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:55.835616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:55.874640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:55.875257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:55.875818 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:55.876580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:55.876966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:55.877271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:55.877271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:55.877590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:55.877980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:55.878575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:55.878575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:55.878575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:56.144663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:56.146823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:56.146961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:56.147956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:56.148293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:56.148524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.149414 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.150519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.150960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:56.151286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.151286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:56.151715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:56.152309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.152864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:56.152864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.152864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:56.190519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:56.191081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:56.191274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:56.191475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:00:56.192218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:56.192538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.193450 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:56.194886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.195376 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.196478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:56.196871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:56.197200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.197200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:56.197200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.197200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:56.199734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.200214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.200719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:56.254663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:56.255383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:56.255782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:56.256199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:56.256922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:56.257442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.258223 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:00:56.259770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.260199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.262516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:56.262939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:56.263321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.263321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:56.263321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.263321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.265242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:56.266042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:00:56.266580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:00:56.266903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:56.267049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:00:56.267270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.268638 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.269821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.270205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:00:56.270543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.270543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:00:56.270747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:00:56.271205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:00:56.271876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:00:56.271876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:00:56.271876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:00.834564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:00.836060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:00.836060 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:00.837075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:00.873499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:00.873678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:00.874323 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:00.875367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:00.875917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:00.876469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:00.876469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:00.877189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:00.877893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:00.878463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:00.878463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:00.878463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:01.147042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:01.147428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:01.147643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:01.148890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:01.149223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:01.149463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.150373 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.152105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.152804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:01.153245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.153245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:01.154452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:01.154913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.155411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:01.155411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.155411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:01.191156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:01.191917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:01.192129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:01.192598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:01.192827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:01.193074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.194182 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:01.195174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.196183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.196667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:01.197232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:01.197660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.197660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:01.197660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.197660 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:01.198462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.198932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.199462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:01.255666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:01.256471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:01.256681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:01.256912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:01.257593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:01.257997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.259151 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:01.261201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.262064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.262527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:01.263120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:01.263729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.263729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:01.263729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.263729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.264531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:01.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:01.266044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.266044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:01.266044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.266044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:01.266488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:01.266827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:01.266938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:01.267094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:01.267400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:01.267650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:01.267833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.268381 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.269292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.269829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:01.270181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.270181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:01.270287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:01.270896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:01.271410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:01.271410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:01.271410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:05.838272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:05.839328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:05.839328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:05.840572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:05.869981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:05.870436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:05.871253 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:05.872035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:05.872476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:05.872947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:05.873445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:05.874149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:05.874149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:05.874149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:05.874149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:05.874149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:05.874149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:06.147699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:06.148100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:06.148299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:06.149339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:06.149593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:06.149772 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.150605 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.152224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:06.152860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.152860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:06.154042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:06.154824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.155492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:06.155492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.155492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:06.192100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:06.192100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:06.192360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:06.192360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:06.193138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:06.193485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.194816 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:06.195151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.195875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.196142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:06.196582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:06.197141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.197676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.198429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:06.256405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:06.256914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:06.256999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:06.257153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:06.257887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:06.258308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.259222 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:06.260110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.260554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.261032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:06.261408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:06.261819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.261819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:06.261819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.261819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.262165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:06.262749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:06.263286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.263286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:06.263286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.263286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:06.266895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:06.267143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:06.267266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:06.267424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:06.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:06.268549 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:06.268967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.269688 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.271097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:06.271446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.271446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:06.271618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:06.272134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:06.272633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:06.272633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:06.272633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:10.839116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:10.840048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:10.840048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:10.841490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:10.875484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:10.875902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:10.876813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:10.877808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:10.878713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:10.879668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:10.880728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:10.882382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:10.882382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:10.882382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:10.882382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:10.882382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:10.882382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:11.148199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:11.148704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:11.148964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:11.150243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:11.150667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:11.151041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.152055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.153793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.154551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:11.155470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:11.156544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.158091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:11.158091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:11.158091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.158091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:11.158091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.158091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:11.192350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:11.192874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:11.193088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:11.193348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:11.194047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:11.194419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.195374 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:11.196631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.197590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.198214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:11.198927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:11.199573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.199573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:11.199573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.199573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:11.199970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.200736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.202075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:11.257204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:11.257917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:11.258247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:11.258504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:11.259598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:11.260006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.260889 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:11.262327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.263099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.264188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:11.264823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:11.265858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.265858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:11.265858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.265858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.266064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:11.266876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:11.268076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:11.268843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:11.269174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:11.269323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.270072 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.270929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.271395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:11.271844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.271844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:11.272112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:11.272612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:11.273155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:11.273155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:11.273155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:15.839935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:15.840854 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:15.840854 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:15.841480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:15.877144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:15.877569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:15.878520 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:15.879508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:15.881191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:15.881976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:15.882608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:15.883442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:15.883442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:15.883442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:15.883442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:15.883442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:15.883442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:16.148889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:16.149559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:16.149870 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:16.151144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:16.151500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:16.151886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.154291 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.155714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.156324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:16.157112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:16.157747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.158633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:16.158633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:16.158633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.158633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:16.158633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.158633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:16.192881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:16.193521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:16.193723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:16.194045 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:16.194788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:16.195157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.196051 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:16.197727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:16.199877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.201373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.201894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:16.202873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.203666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.204446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:16.257869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:16.258420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:16.258643 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:16.258984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:16.259797 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:16.260224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.261497 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:16.262048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.262629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:16.264147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:16.264914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:16.265628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.266364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:16.267839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:16.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:16.268253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:16.268440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:16.268902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:16.269250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:16.269482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.270200 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.271110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:16.271773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.271773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:16.272306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:16.272779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:16.273257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:16.273257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:16.273257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:20.840357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:20.841002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:20.841002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:20.842236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:20.844267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=434 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:01:20.885368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:20.885583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:20.886175 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:20.887117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:20.887859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:20.888579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:20.889448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:20.890478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:20.890478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:20.890478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:20.890478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:20.890478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:20.890478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:21.149607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:21.149911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:21.150097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:21.151182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:21.151516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:21.151787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.152650 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.153705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.154306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:21.154873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:21.155531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.156348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:21.156348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:21.156348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.156348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:21.156348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.156348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:21.193542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:21.194006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:21.194206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:21.194403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:21.195146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:21.195521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.196406 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:21.197746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.198567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.199007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:21.199689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:21.200318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.200947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.201716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:21.258544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:21.259055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:21.259316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:21.259672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:21.260487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:21.261039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.262055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:21.263778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.264413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.264816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:21.265369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:21.265988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:21.266620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.267433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:21.269266 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:21.269577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:21.269692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:21.269918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:21.270362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:21.270683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:21.270919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.271597 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.272434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.272864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:21.273275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.273275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:21.273453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:21.273973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:21.274521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:21.274521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:21.274521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:25.840853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:25.842041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:25.842041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:25.843143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:25.878745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:25.879088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:25.880021 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:25.880977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:25.881637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:25.882318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:25.883179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:25.884311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:25.884311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:25.884311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:25.884311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:25.884311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:25.884311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:26.150081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:26.150864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:26.151154 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:26.152392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:26.152772 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:26.153090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.154321 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.155923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.156312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:26.156666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.156666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:26.158114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:26.158911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.159778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:26.159778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.159778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:26.194070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:26.194524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:26.194650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:26.194808 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:26.195416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:26.195783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.196561 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:26.197476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:26.198369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.199157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.199861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:26.200444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.201559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:26.259234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:26.259669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:26.259846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:26.259946 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:26.260580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:26.261012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.261914 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:26.262617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.263405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.263855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:26.264395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:26.264977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:26.266492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.268118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:26.269833 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:26.270355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:26.270578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:26.271323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:26.271911 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:26.272294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:26.272539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.273399 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.275093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.275652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:26.276095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.276095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:26.276411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:26.277051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:26.277803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:26.277803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:26.277803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:30.841922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:30.842696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:30.842696 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:30.844125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:30.876438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:30.876889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:30.878185 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:30.878664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:30.879344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:30.879983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:30.880738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:30.881657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:30.881657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:30.881657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:30.881657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:30.881657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:30.881657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:31.150715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:31.151169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:31.151740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:31.152553 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:31.152978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:31.153216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.154115 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.155691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.156165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:31.156543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.156543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:31.157389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:31.157931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:31.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.158475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:31.194563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:31.194920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:31.195047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:31.195186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:31.196019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:31.196393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.197130 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:31.198027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.198783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.199224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:31.199846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:31.200413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.200413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:31.200413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.200413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:31.200581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.201248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.202287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:31.259811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:31.260140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:31.260140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:31.261302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:31.261302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:31.261514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.262381 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:31.263093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:31.264159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:31.264859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:31.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:31.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.266640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:31.270417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:31.270754 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:31.270942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:31.271169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:31.274066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:31.274327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:31.274476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.275125 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.276282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.276709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:31.277152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.277152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:31.277640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:31.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:31.279316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:31.279316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:31.279316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:35.842681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:35.843396 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:35.843543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:35.844903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:35.884998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:35.885247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:35.886269 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:35.887481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:35.887968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:35.888393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:35.888393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:35.888578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:35.889131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:35.889702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:35.889702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:35.889702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:36.151268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:36.151742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:36.151941 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:36.153117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:36.153475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:36.153754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.154725 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.155974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.156550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:36.157247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:36.157943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.158799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:36.158799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:36.158799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.158799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:36.158799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.158799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:36.195257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:36.195939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:36.196199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:36.196437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:36.197277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:36.197703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.198654 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:36.200249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.201098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.201618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:36.202253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:36.202894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.202894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:36.202894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.202894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:36.203399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.204106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.204615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:36.260389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:36.260876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:36.261065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:36.261275 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:36.261923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:36.262355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.263158 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:36.264289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.265077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.265500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:36.266467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:36.267421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:36.268260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.269309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:36.271057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:36.271589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:36.271836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:36.272099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:36.272542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:36.272970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:36.273201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.273962 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.275442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.275841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:36.276145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.276145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:36.277144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:36.277586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:36.278111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:36.278111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:36.278111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:40.843274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:40.843883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:40.844451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:40.846064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:40.888008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:40.888384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:40.889091 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:40.892158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:40.892511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:40.892749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:40.893065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:40.893425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:40.893425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:40.893425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:40.893425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:40.893425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:40.893425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:41.152074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:41.152599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:41.152844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:41.153952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:41.154325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:41.154577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.155396 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.157892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.158506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:41.159252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:41.159694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.160209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:41.160209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:41.160209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.160209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:41.160209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.160209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:41.196001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:41.196531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:41.196653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:41.196782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:41.197320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:41.197542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.198192 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:41.199173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.199602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.200082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:41.200501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:41.200836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.200836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:41.200836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.200836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:41.201184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.201552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.202570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:41.260914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:41.261262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:41.261568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:41.261650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:41.262318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:41.262619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.263280 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:41.264204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.265110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.266168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:41.266985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:41.267675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:41.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.269621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:41.271442 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:41.271776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:41.271900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:41.272152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:41.272499 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:41.272776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:41.272963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.273484 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.274361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.274779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:41.275158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.275158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:41.275401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:41.275939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:41.276503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:41.276503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:41.276503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:45.844143 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:45.844979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:45.844979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:45.845314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:45.872121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:45.872261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:45.872713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:45.873393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:45.874159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:45.875231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:45.876289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:45.877340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:45.877340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:45.877340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:45.877340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:45.877340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:45.877340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:46.152514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:46.153030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:46.153175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:46.154020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:46.154207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:46.154355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.155209 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.156149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.156723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:46.157335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.157335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:46.159026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:46.159852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.160574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:46.160574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.160574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:46.196748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:46.197249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:46.197357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:46.197503 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:46.198084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:46.198436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.199026 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:46.199812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:46.201064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.202171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.203150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:46.203949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.205167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.205926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:46.261514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:46.262133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:46.262323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:46.262525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:46.263325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:46.263783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.264646 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:46.265962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.266928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.267463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:46.268330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:46.269869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:46.270752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.271954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:46.272616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:46.272616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:46.272616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:46.272616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:46.273116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:46.273516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:46.273764 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.274521 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.275362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.275711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:46.276085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.276085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:46.276330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:46.276739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:46.277281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:46.277281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:46.277281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:50.845449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:50.846159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:50.846394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:50.848402 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:50.885315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:50.885630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:50.886358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:50.887109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:50.887901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:50.888649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:50.889549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:50.890266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:50.890266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:50.890266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:50.890266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:50.890266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:50.890266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:51.153056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:51.153492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:51.153706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:51.155269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:51.155687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:51.156024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.157100 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.158183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.158871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:51.159569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:51.160332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.161266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:51.161266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:51.161266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.161266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:51.161266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.161266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:51.197491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:51.198720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:51.198720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:51.198720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:51.199786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:51.200377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.200965 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:51.202264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.202719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.203654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:51.204070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:51.204421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.204421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:51.204421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.204421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:51.205433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.205773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.206119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:51.262133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:51.262441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:51.262550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:51.262730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:51.263448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:51.263802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.264707 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:51.265720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.266163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.266855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:51.267220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:51.267591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.267591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:51.267591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.267591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.268339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:51.268999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:51.269473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.269473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:51.269473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.269473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:51.272440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:51.273157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:51.273329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:51.273586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:51.274144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:51.274463 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:51.274665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.275437 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.276781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.277431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:51.278140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:51.279073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:51.280364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:51.280364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:51.280364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:51.280364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:51.280364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:51.280364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:55.846311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:55.847219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:55.847219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:55.848385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:55.884223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:55.884722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:55.885995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:55.887910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:55.888801 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:55.889642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:55.889642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:55.890078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:55.890979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:55.891832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:55.891832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:55.891832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:56.153647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:56.153899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:56.153899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:56.154617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:56.155027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:56.155301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.156936 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.157593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.158038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:56.158464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:56.158983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.159900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:56.159900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:56.159900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.159900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:56.159900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.159900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:56.198142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:56.198418 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:56.198568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:56.198694 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:01:56.199371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:56.199781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.200834 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:56.201647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.202481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.202974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:56.203623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:56.204308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.205076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.206104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:56.262739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:56.263089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:56.263222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:56.263351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:56.263924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:56.264432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.265230 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:01:56.266118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:56.267031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:56.268052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:56.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:01:56.269119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.269622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:01:56.272938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:01:56.273824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:01:56.274031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:01:56.274316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:01:56.274659 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:56.274914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:01:56.275059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.275668 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.277269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.277669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:01:56.278065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.278065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:01:56.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:01:56.279439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:01:56.280057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:01:56.280057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:01:56.280057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:00.847361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:00.848368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:00.848368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:00.849464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:00.882639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:00.883128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:00.884028 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:00.884909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:00.885597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:00.886468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:00.887309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:00.889207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:00.889207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:00.889207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:00.889207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:00.889207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:00.889207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:01.154260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:01.154778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:01.154964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:01.156127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:01.156558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:01.156903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.157914 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.159691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.160788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:01.161837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:01.162295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.162830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:01.162830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:01.162830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.162830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:01.162830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.162830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:01.198851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:01.199315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:01.199489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:01.199646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:01.200375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:01.200701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.201523 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:01.202530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.203640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.204051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:01.204995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:01.206285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.207308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.208084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:01.263429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:01.264146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:01.264810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:01.265258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:01.266081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:01.266551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.267554 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:01.269143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.271244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:01.271973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:01.272678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.272678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:01.272678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.272678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.273190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:01.273863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:01.274539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:01.275037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:01.275436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:01.275715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.276338 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.277329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.277907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:01.278473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.278473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:01.278722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:01.279293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:01.279803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:01.279803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:01.279803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:05.847874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:05.849202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:05.849202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:05.849834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:05.887596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:05.888141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:05.888983 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:05.889788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:05.890524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:05.891263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:05.892145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:05.892631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:05.892631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:05.892631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:05.892631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:05.892631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:05.892631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:06.154935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:06.155159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:06.155273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:06.156231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:06.156561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:06.156832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.157604 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.158318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.158792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:06.159260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:06.159710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.160328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:06.160328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:06.160328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.160328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:06.160328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.160328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:06.199341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:06.199853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:06.200153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:06.200400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:06.200969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:06.201279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.202103 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:06.203427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.204205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.205336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:06.206067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:06.206716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.206716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:06.206716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.206716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:06.207684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.208551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.209331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:06.264186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:06.264663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:06.264906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:06.265118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:06.265711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:06.266159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.266992 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:06.268230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.268710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.269695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:06.270079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:06.270525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.270525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:06.270525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.270525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.271901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:06.272636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:06.273274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.273274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:06.273274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.273274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:06.274054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:06.274423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:06.274613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:06.274897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:06.275195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:06.275410 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:06.275552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.276055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.276881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.277289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:06.277710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.277710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:06.277858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:06.278374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:06.278974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:06.278974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:06.278974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:10.848557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:10.849459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:10.849459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:10.850599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:10.885493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:10.885967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:10.886845 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:10.887686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:10.888697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:10.889332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:10.890211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:10.893748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:10.893748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:10.893748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:10.893748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:10.893748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:10.893748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:11.155827 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:11.156551 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:11.156720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:11.157966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:11.158360 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:11.158616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.159708 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.161626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.162812 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:11.163557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:11.164506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.166392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:11.166392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:11.166392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.166392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:11.166392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.166392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:11.199995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:11.200594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:11.200828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:11.200994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:11.201600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:11.201896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.202735 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:11.203657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.204297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.204674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:11.205403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:11.206270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.206989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.207982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:11.264718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:11.265399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:11.265607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:11.265852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:11.266669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:11.267129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.267789 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:11.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.270074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.271104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:11.271439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:11.271819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.271819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:11.271819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.271819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.273479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:11.274036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:11.274574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.274574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:11.274574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.274574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:11.274574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:11.275206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:11.275330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:11.275491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:11.275836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:11.276147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:11.276466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.277194 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.278350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.278750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:11.279124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.279124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:11.279463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:11.280037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:11.280575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:11.280575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:11.280575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:15.849068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:15.850138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:15.850138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:15.851432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:15.888664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:15.889115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:15.889971 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:15.890940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:15.891479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:15.892090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:15.892781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:15.893586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:15.893586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:15.893586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:15.893586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:15.893586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:15.893586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:16.156861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:16.157510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:16.157914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:16.159140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:16.159506 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:16.159985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.160874 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.162805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.163522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:16.164210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.164210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:16.164644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:16.166894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.168595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:16.168595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.168595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:16.200785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:16.201244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:16.201420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:16.201561 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:16.202310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:16.202777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.203731 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:16.205290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.205619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.206929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:16.207617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:16.208223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.208223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:16.208223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.208223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:16.209097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.210063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.211021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:16.265425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:16.266039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:16.266273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:16.266469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:16.267081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:16.267504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.268379 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:16.269673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.270707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.271330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:16.272191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:16.273222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:16.274187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:16.275352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:16.276213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:16.276213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:16.276213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:16.276479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:16.276802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.277388 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.278183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.278542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:16.278950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.278950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:16.279149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:16.279601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:16.280165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:16.280165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:16.280165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:20.849641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:20.850276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:20.850486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:20.851908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:20.858993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=241 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:02:20.884479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:20.885003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:20.885953 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:20.886714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:20.887656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:20.888372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:20.889262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:20.890293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:20.890293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:20.890293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:20.890293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:20.890293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:20.890293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:21.157651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:21.158122 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:21.158411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:21.159723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:21.160206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:21.160475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.161471 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.162720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:21.163380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.163380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:21.163884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:21.164311 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.164745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:21.164745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.164745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:21.201361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:21.201578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:21.201706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:21.201957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:21.202462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:21.202964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.204046 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:21.205418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.205902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.207881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:21.208208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:21.208526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.208526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:21.208526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.208526 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:21.208884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.209315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.209749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:21.266171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:21.266839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:21.267044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:21.267255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:21.267958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:21.268440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.269136 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:21.270479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.270875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.272331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:21.272663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:21.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:21.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.273017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.274508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:21.274976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:21.275400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.275400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:21.275400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.275400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:21.275507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:21.275859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:21.275964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:21.276114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:21.276406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:21.276670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:21.276878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.277348 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.278157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.278484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:21.278814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.278814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:21.279228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:21.279661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:21.280145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:21.280145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:21.280145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:25.850474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:25.851305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:25.851305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:25.852621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:25.881468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:25.881774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:25.882409 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:25.883171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:25.883910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:25.884499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:25.885357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:25.886241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:25.886241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:25.886241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:25.886241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:25.886241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:25.886241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:26.158252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:26.159354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:26.159554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:26.160792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:26.161225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:26.161545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.162514 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.164260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.164884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:26.165479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.165479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:26.166105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:26.166789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.167550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:26.167550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.167550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:26.201901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:26.202249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:26.202518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:26.202795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:26.203550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:26.203963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.205097 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:26.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.207593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.208522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:26.209179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:26.209895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.209895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:26.209895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.209895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.211153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.211909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:26.266816 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:26.267123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:26.267298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:26.267497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:26.268286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:26.268983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.270651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:26.272179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.273033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.273850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:26.274588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:26.275280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.275280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:26.275280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.275280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.276733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:26.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:26.278720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:26.279405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:26.279405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:26.279405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.279995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.280696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.281134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:26.281576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:26.282079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:26.282643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:26.282643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:26.282643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:26.282643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:26.282643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:26.282643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:30.851292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:30.852105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:30.852105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:30.853313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:30.882642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:30.883187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:30.884045 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:30.884738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:30.885190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:30.885680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:30.886275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:30.886869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:30.886869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:30.886869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:30.886869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:30.886869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:30.886869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:31.159533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:31.160262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:31.160482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:31.161667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:31.162082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:31.162342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.163318 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.165282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.168212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:31.168548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:31.169007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.169613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:31.169613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:31.169613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.169613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:31.169613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.169613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:31.202469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:31.202990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:31.203351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:31.203474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:31.204247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:31.204556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.205493 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:31.207190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.207971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.208919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:31.209465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:31.209933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.209933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:31.209933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.209933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:31.210400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.211251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:31.267509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:31.268150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:31.268398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:31.269633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:31.270313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.271914 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:31.272697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.273652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.274203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:31.275497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:31.276543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:31.277998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:31.279054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:31.279523 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:31.279822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.279822 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:31.279822 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.279822 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.280026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.280408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:31.280776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.280776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:31.280978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:31.281642 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:31.282236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:31.282236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:31.282236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:35.885173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:35.885173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:35.885173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:35.885173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:35.886237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:35.886237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:35.889786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:35.890638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:35.891072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:35.891831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:35.892430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:36.160341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:36.160836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:36.161019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:36.162307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:36.162711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:36.163008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.163959 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.165614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.166633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:36.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:36.167528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:36.168114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.168458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:36.168458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.168458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:36.203084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:36.203492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:36.203574 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:36.203737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:36.204340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:36.204673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.205597 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:36.206923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.207323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.208276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:36.208638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:36.209011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.209011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:36.209011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.209011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:36.210691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.211286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.211785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:36.268955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:36.268955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:36.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:36.269615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:36.270878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:36.270878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.271916 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:36.273701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.274433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.274858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:36.275434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:36.276133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:36.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:36.279512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:36.280479 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.281088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:36.282910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:36.283727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:36.284696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:36.285185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:36.285185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:36.285185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:36.285185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:36.285185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:36.285185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:40.852480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:40.853394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:40.853394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:40.854525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:40.883409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:40.883683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:40.884400 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:40.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:40.886139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:40.887106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:40.888272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:40.889832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:40.889832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:40.889832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:40.889832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:40.889832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:40.889832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:41.161033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:41.161597 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:41.161832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:41.162994 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:41.163366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:41.163609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.164498 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.166094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.166483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:41.167123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.167123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:41.167808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:41.168611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.169694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:41.169694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.169694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:41.204180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:41.206458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:41.206640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:41.206865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:41.207629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:41.207966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.208678 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:41.209968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.210480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:41.211695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:41.212222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.212222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:41.212222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.212222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:41.213054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.213655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.214204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:41.268890 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:41.269400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:41.269552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:41.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:41.270458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:41.271010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.271912 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:41.272819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:41.273931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:41.274586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:41.275405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:41.276061 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.277068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:41.278095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:41.278364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:41.278544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:41.278693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:41.279004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:41.279254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:41.279387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.279942 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.280786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.281194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:41.281467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.281467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:41.281856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:41.282309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:41.282628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:41.282628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:41.282628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:45.853020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:45.853955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:45.853955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:45.855320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:45.892281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:45.892647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:45.893549 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:45.894324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:45.895209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:45.896112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:45.897010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:45.898118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:45.898118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:45.898118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:45.898118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:45.898118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:45.898118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:46.161686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:46.162257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:46.162358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:46.163183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:46.163569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:46.163860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.164625 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.165383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.165705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:46.166043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.166043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:46.166143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:46.166751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.167224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:46.167224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.167224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:46.206729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:46.207207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:46.207391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:46.207579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:46.208161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:46.208495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.209568 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:46.210457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.210930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.211633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:46.211996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:46.212525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.212525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:46.212525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.212525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:46.213015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.213429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.213911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:46.270258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:46.270451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:46.270607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:46.270745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:46.271130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:46.271389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.271939 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:46.272672 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.273041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.273559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:46.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:46.274205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.274205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:46.274205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.274205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.274679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:46.275147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:46.275601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.275601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:46.275601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.275601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:46.278562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:46.278861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:46.278978 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:46.279131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:46.279546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:46.279953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:46.280197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.280958 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.281731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.282075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:46.282447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.282447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:46.282730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:46.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:46.283617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:46.283617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:46.283617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:50.854392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:50.855249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:50.855316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:50.856930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:50.883612 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:50.884236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:50.885247 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:50.886025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:50.886641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:50.887350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:50.887735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:50.888252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:50.888252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:50.888252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:50.888252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:50.888252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:50.888252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:51.162593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:51.163086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:51.163381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:51.164023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:51.164401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:51.164630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.165656 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.166653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.167243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:51.168165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:51.168853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.169646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:51.169646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:51.169646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.169646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:51.169646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.169646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:51.207371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:51.207846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:51.208028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:51.208225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:51.209121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:51.209379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.210081 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:51.211276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:51.212610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.213446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.213781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:51.214780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.215874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.216573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:51.270810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:51.271188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:51.271393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:51.271627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:51.272206 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:51.272490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.273160 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:51.274528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.275108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.276117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:51.276457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:51.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:51.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.276959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.277995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:51.279074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:51.279992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:51.280225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:51.280501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:51.280501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.281125 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.282152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.282638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:51.283140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.283140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:51.283655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:51.284558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:51.285543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:51.285543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:51.285543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:55.855286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:55.856511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:55.856511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:55.857320 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:55.889099 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:55.889302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:55.890112 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:55.891133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:55.891484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:55.891841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:55.891841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:55.891909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:55.892326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:55.892829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:55.892829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:55.892829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:56.163596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:56.164372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:56.164654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:56.166130 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:56.166647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:56.167049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.168240 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.170303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.170925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:56.171496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.171496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:56.172270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:56.173065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.173926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:56.173926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.173926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:56.208034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:56.208595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:56.208693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:56.208883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:02:56.209527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:56.210006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.210947 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:56.211971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.212831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.213301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:56.213947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:56.214625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.215323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.216281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:56.271565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:56.272223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:56.272454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:56.272700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:56.273322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:56.273721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.274641 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:02:56.275941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.276692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.277937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:56.278547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:56.279320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.279320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:56.279320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.279320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.279937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:56.280480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:02:56.281064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:02:56.281341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:56.281603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:02:56.281912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.282514 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.284213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.284702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:02:56.285122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.285122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:02:56.285309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:02:56.285853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:02:56.286456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:02:56.286456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:02:56.286456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:00.856349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:00.857321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:00.857321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:00.858323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:00.884566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:00.885168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:00.886201 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:00.887031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:00.887455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:00.888065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:00.888065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:00.888230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:00.888724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:00.889297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:00.889297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:00.889297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:01.164372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:01.164880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:01.165081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:01.166021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:01.166348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:01.166482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.167383 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.168945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.169803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:01.170356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.170356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:01.170623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:01.171075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.171609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:01.171609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.171609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:01.208938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:01.209422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:01.209623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:01.209873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:01.210386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:01.210682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.211502 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:01.212750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.213524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.214189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:01.214598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:01.215034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.215034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:01.215034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.215034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:01.215356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.215835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.216343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:01.272340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:01.272912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:01.273113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:01.273323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:01.274097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:01.274789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.275849 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:01.277094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.277851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.279001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:01.279602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:01.280467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:01.281143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:01.282125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:01.282924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:01.282924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:01.282924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:01.282924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:01.282924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.283776 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.284610 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.285171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:01.285661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:01.286255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:01.287046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:01.287046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:01.287046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:01.287046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:01.287046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:01.287046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:05.856989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:05.857916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:05.857916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:05.859034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:05.896699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:05.897013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:05.897750 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:05.898680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:05.899674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:05.900095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:05.900639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:05.901264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:05.901264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:05.901264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:05.901264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:05.901264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:05.901264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:06.164824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:06.165471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:06.165641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:06.166374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:06.166607 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:06.166776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.170445 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.172305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.172746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:06.173152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.173152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:06.174135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:06.174633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.175190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:06.175190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.175190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:06.209746 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:06.210393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:06.210591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:06.210819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:06.211719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:06.212290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.213470 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:06.214926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.215673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.216445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:06.217082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:06.217737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.217737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:06.217737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.217737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:06.218275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.218894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.219449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:06.273087 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:06.273793 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:06.274030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:06.274238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:06.274901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:06.275333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.276161 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:06.277169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.277922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.278310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:06.279045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:06.279684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:06.280454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:06.281393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:06.282017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:06.282017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:06.282017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:06.282222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:06.282406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:06.282553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.283157 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.284198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.284849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:06.285234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.285234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:06.285401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:06.285893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:06.286453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:06.286453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:06.286453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:10.857716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:10.858416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:10.858684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:10.860039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:10.905290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:10.906106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:10.907272 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:10.909035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:10.909601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:10.910051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:10.910051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:10.910693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:10.911200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:10.911753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:10.911753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:10.911753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:11.165416 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:11.178255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:11.178493 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:11.179681 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:11.180194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:11.180516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.181880 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.185832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.186287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:11.186646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:11.187126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.188632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:11.188632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:11.188632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.188632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:11.188632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.188632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:11.210534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:11.211074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:11.211201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:11.211351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:11.211967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:11.212192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.213168 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:11.214427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.214931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.216266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:11.216628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:11.217158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.217158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:11.217158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.217158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:11.219092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.219848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.220327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:11.274147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:11.274868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:11.275103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:11.275323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:11.276160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:11.276659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.277813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:11.279212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.279789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.281528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:11.281815 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:11.282180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.282180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:11.282180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.282180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:11.282180 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.283229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:11.283598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:11.284043 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:11.284270 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:11.284312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:11.284440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.284922 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.285907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.286229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:11.286552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.286552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:11.286930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:11.287300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:11.287633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:11.287633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:11.287633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:15.859028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:15.860084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:15.860084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:15.861287 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:15.894970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:15.895375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:15.896335 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:15.897379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:15.898050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:15.898692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:15.899488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:15.900535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:15.900535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:15.900535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:15.900535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:15.900535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:15.900535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:16.178199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:16.178901 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:16.179191 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:16.180481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:16.180541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:16.180793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.181651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.183292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.183670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:16.184032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.184032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:16.184373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:16.184837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.185331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:16.185331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.185331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:16.211259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:16.211705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:16.211941 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:16.212124 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:16.212692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:16.213043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.213909 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:16.215187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.216137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.216821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:16.217704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:16.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:16.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:16.219098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.219992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.220905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:16.275104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:16.275641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:16.275853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:16.276116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:16.276750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:16.277285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.278112 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:16.279055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.279518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.279983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:16.280263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:16.280530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.280530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:16.280530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.280530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.281254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:16.281600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:16.282049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.282049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:16.282049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.282049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:16.283623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:16.283969 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:16.284094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:16.284248 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:16.284700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:16.285393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:16.285652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.286318 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.287201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.287656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:16.288093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.288093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:16.288159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:16.288651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:16.289254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:16.289254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:16.289254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:20.859897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:20.860794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:20.860794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:20.861525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:20.869686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=241 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:03:20.892423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:20.892701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:20.893516 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:20.894309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:20.894679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:20.895019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:20.895019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:20.895309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:20.896141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:20.897270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:20.897270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:20.897270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:21.179125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:21.179600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:21.179880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:21.180907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:21.180907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:21.181072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.181707 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.185416 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.186120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:21.186615 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:21.187029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.187485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:21.187485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:21.187485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.187485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:21.187485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.187485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:21.211802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:21.212239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:21.212239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:21.212403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:21.212670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:21.212855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.213273 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:21.214005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.214790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.215364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:21.216234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:21.217004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.217263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.219691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:21.275664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:21.276326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:21.276438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:21.276655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:21.277395 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:21.277665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.278293 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:21.279345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.279914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.280712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:21.282195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:21.282443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:21.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.283267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:21.284164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:21.284739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:21.284888 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:21.285068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:21.285690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:21.285973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:21.286165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.287093 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.288618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.289037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:21.289395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.289395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:21.289730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:21.290918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:21.291970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:21.291970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:21.291970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:25.862207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:25.862207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:25.862207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:25.862207 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:25.895768 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:25.896299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:25.897131 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:25.898353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:25.899035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:25.899635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:25.899635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:25.899750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:25.900510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:25.902195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:25.902195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:25.902195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:26.179716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:26.180153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:26.180272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:26.181790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:26.182164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:26.182589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.183491 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.184288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.185021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:26.185359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:26.185786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.186282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:26.186282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:26.186282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.186282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:26.186282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.186282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:26.213200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:26.213200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:26.213200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:26.213200 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:26.213801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:26.214565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.214984 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:26.215616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:26.216874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.217996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.218752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:26.219512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.220592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.221473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:26.276224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:26.278168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:26.278273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:26.278379 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:26.279125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:26.279625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.280525 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:26.281208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:26.282155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:26.283136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:26.284003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:26.284723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.286003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:26.287252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:26.287252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:26.287252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:26.287252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:26.287887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:26.288251 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:26.288468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.289018 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.290060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:26.290420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:26.290902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:26.291442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:26.291442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:26.291442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:26.291442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:26.291442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:26.291442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:30.862242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:30.862242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:30.862242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:30.863012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:30.899862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:30.900157 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:30.900832 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:30.901634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:30.901985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:30.902236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:30.902236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:30.902573 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:30.906681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:30.907115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:30.907115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:30.907115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:31.180255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:31.180862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:31.181092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.184342 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.184883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.185606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:31.185980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.185980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:31.186875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:31.187354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.187876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:31.187876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.187876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:31.212859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:31.213288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:31.213448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:31.213564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:31.214280 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:31.214567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.215166 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:31.215946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:31.216622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.217048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.217454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:31.217787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.218221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.218593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:31.278329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:31.278891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:31.279079 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:31.279326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:31.279993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:31.280258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.281100 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:31.282456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:31.286129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:31.286985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:31.288425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:31.289283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.290710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:31.291661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:31.291661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:31.291661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:31.291661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:31.291661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:31.291661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.292022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.292482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:31.293053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:31.293709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:31.294616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:35.861970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:35.862845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:35.862845 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:35.864237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:35.900272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:35.900602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:35.901454 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:35.902246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:35.902588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:35.902951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:35.902951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:35.903297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:35.904271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:35.905340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:35.905340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:35.905340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:36.180965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:36.181404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:36.181600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:36.182701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:36.183075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:36.183325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.184221 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.185712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.186423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:36.186956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.186956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:36.188047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:36.188697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.189448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:36.189448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.189448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:36.213344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:36.213703 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:36.213933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:36.214113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:36.215182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:36.215521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.216467 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:36.218109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:36.219018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.219629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.220209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:36.220701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.221368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.222663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:36.278967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:36.279422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:36.279621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:36.280003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:36.280598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:36.281066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.281992 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:36.283218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.284261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.284874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:36.285565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:36.286450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.286450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:36.286450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.286450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.286709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:36.287716 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:36.288897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:36.289440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:36.289440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:36.289440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.289994 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.291149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.291454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:36.291805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.291805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:36.292336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:36.292708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:36.293198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:36.293198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:36.293198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:40.862698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:40.863661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:40.863661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:40.864905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:40.898121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:40.898554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:40.899370 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:40.899957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:40.900393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:40.900842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:40.901308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:40.901906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:40.901906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:40.901906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:40.901906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:40.901906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:40.901906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:41.181655 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:41.182446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:41.182861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:41.184137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:41.184444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:41.184670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.185505 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.186652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.187446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:41.188404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:41.189141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.190250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:41.190250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:41.190250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.190250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:41.190250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.190250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:41.213932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:41.214306 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:41.214435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:41.214594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:41.215090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:41.215466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.216134 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:41.217033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.217740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.218148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:41.218703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:41.219306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.219970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.220817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:41.279432 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:41.280025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:41.280256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:41.280500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:41.287338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:41.291537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:41.292137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.293166 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:41.295911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.014 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:41.296599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:41.297191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.297720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.298323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:41.299491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:41.300330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:41.300684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:41.300684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.300906 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.302200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.302867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:41.303199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:41.303846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:41.304232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:41.304232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:41.304232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:41.304232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:41.304232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:41.304232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:45.863504 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:45.864444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:45.864444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:45.865425 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:45.899072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:45.899375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:45.900250 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:45.901363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:45.901859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:45.902326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:45.902326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:45.902528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:45.903730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:45.904910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:45.904910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:45.904910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:46.182439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:46.183030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:46.183157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:46.183817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:46.184032 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:46.184190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.184878 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.185840 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.186270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:46.186628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.186628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:46.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:46.187400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.187912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:46.187912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.187912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:46.214524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:46.215260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:46.215260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:46.215485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:46.215886 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:46.216171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.216956 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:46.217745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.218463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.218863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:46.219353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:46.220403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.221449 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.222363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:46.280237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:46.280737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:46.280882 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:46.280987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:46.281588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:46.282007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.282817 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:46.283479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.284165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.284581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:46.285120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:46.285870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:46.286606 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.287657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:46.298638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:46.299095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:46.299259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:46.299492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:46.300052 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:46.300838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:46.300838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.301495 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.302630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.303084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:46.303504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.303504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:46.304005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:46.304508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:46.305080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:46.305080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:46.305080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:50.864383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:50.865303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:50.865303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:50.866064 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:50.897066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:50.897451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:50.898373 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:50.899174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:50.899893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:50.900592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:50.901498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:50.902444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:50.902444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:50.902444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:50.902444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:50.902444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:50.902444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:51.183449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:51.183869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:51.184003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:51.185084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:51.185524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:51.186011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.186495 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.188238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.188701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:51.189242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.189242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:51.190200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:51.190912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.191465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:51.191465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.191465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:51.215250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:51.215907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:51.216166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:51.216305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.218001 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:51.219485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.220240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.220711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:51.221202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:51.221561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.221561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:51.221561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.221561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:51.221928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.222408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.222956 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:51.280899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:51.281505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:51.281840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:51.281965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:51.282958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.283733 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:51.285397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.285942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.286933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:51.287326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:51.288046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.288046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:51.288046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.288046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.288885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:51.289394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:51.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:51.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.289955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:51.299216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:51.299489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:51.299581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:51.299744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:51.300294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:51.300738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:51.300938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.302990 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.304411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.305140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:51.305490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.305490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:51.306053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:51.306568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:51.307087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:51.307087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:51.307087 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:55.865282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:55.866055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:55.866055 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:55.866737 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:55.907497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:55.907935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:55.908732 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:55.909837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:55.910511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:55.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:55.912035 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:55.912687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:55.912687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:55.912687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:55.912687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:55.912687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:55.912687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:56.183937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:56.184210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:56.184327 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:56.185209 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:56.185448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:56.185601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.186197 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.187397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.187751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:56.188144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.188144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:56.188435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:56.188880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.189400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:56.189400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.189400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:56.216068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:56.216358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:56.216487 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:56.216620 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:03:56.217331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:56.217457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.218104 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:56.219025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.219621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.219990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:56.220461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:56.221032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.221595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.222415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:56.281807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:56.282203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:56.282388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:56.282581 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:56.283434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:56.283933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.284748 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:03:56.286198 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.286884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.287707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:56.288184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:56.288586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.288586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:56.288586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.288586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.289549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:56.290077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:56.290627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.290627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:56.290627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.290627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:03:56.299749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:03:56.300255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:03:56.300438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:03:56.300725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:03:56.301433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:56.301685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:03:56.301871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.302547 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.304845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:03:56.305368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.305368 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:03:56.306334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:03:56.307049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:03:56.307811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:03:56.307811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:03:56.307811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:00.866091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:00.867091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:00.867091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:00.867962 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:00.904312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:00.904882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:00.906067 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:00.907779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:00.908491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:00.909265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:00.909265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:00.910121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:00.910726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:00.911415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:00.911415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:00.911415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:01.184657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:01.185153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:01.185348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:01.186509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:01.186924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:01.187175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.188092 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.189689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.190151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:01.190533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.190533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:01.191643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:01.192374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.192989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:01.192989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.192989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:01.216615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:01.216889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:01.217076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:01.217174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:01.217777 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.218710 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:01.219836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:01.220970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.221686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.222331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:01.222931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.223636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.224599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:01.282449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:01.282699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:01.282824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:01.282983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:01.283757 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:01.284317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.285340 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:01.286163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.286824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.287211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:01.287701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:01.288317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:01.288976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.289915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:01.300223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:01.300491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:01.300588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:01.300778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:01.301198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:01.301544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:01.301741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.302477 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.304128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.304488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:01.304904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.304904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:01.305907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:01.306357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:01.306917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:01.306917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:01.306917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:05.866485 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:05.867218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:05.867218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:05.867688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:05.903247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:05.903563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:05.904640 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:05.905782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:05.906799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:05.907436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:05.907436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:05.907605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:05.908520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:05.909411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:05.909411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:05.909411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:06.185189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:06.185454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:06.185559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:06.186138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:06.186317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:06.186466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.187083 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.187778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.188199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:06.188455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.188455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:06.188569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:06.189025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.189564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:06.189564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.189564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:06.218155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:06.218428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:06.218428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:06.218569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:06.219337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:06.219729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.220589 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:06.221261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:06.222022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.222533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.223039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:06.223461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.224057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.224595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:06.282967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:06.283405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:06.283716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:06.283810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:06.284676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:06.284983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.286344 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:06.287295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.287827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.288295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:06.288841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:06.289233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.289233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:06.289233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.289233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.289403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:06.289933 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:06.290441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.290441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:06.290441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.290441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:06.300788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:06.301674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:06.301674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:06.301674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:06.302342 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:06.302714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:06.302935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.303822 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.305057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.305680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:06.306099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.306099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:06.306462 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:06.307236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:06.309384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:06.309384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:06.309384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:10.867077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:10.868028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:10.868028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:10.868451 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:10.900895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:10.903056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:10.904077 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:10.905347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:10.905848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:10.906250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:10.906250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:10.906819 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:10.907318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:10.908008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:10.908008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:10.908008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:11.185850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:11.186945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.203354 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.203857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.204241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:11.204632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:11.206911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.209574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:11.209574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:11.209574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.209574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:11.209574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.209574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:11.218457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:11.218942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:11.219305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:11.219305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:11.219799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.221064 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:11.221776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.222189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.222737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:11.224310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:11.224781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.224781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:11.224781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.224781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:11.225602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.226070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.226569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:11.283674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:11.284449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:11.284640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:11.284877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:11.285638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:11.285780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.286641 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:11.288290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.288708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.289187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:11.290277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:11.291036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:11.291424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.291916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:11.302317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:11.302846 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:11.303098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:11.303353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:11.303840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:11.304224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:11.304455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.305176 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.305857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.306270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:11.306630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:11.307052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:11.307548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:11.307548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:11.307548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:11.307548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:11.307548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:11.307548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:15.867675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:15.868471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:15.868471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:15.870312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:15.904486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:15.904661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:15.905534 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:15.906363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:15.907267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:15.908187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:15.909293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:15.910797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:15.910797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:15.910797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:15.910797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:15.910797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:15.910797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:16.186789 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:16.187132 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:16.187331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:16.187953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:16.188127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:16.188277 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.188871 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.190121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.190945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:16.192324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:16.193144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.193827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:16.193827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:16.193827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.193827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:16.193827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.193827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:16.218953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:16.219231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:16.219326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:16.219458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:16.219878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:16.220107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.220671 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:16.222065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.222842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.223381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:16.224588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:16.225082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.225082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:16.225082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.225082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:16.225364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.226378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.226830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:16.284474 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:16.285030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:16.285286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:16.285592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:16.286368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:16.286687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.287806 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:16.289400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.290097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.290968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:16.291495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:16.292082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.292082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:16.292082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.292082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.292972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:16.293752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:16.295149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.295149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:16.295149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.295149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:16.302719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:16.302984 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:16.303105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:16.303267 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:16.303879 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:16.304369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:16.304686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.305538 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.306693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.307791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:16.308443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:16.309149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:16.309947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:16.309947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:16.309947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:16.309947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:16.309947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:16.309947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:20.868469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:20.869356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:20.869356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:20.870465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:20.877633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=239 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:04:20.913078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:20.913476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:20.914447 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:20.915603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:20.916246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:20.917132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:20.917818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:20.918594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:20.918594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:20.918594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:20.918594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:20.918594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:20.918594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:21.187422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:21.188020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:21.188224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:21.189125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:21.189343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:21.189436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.190140 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.191686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.192387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:21.193120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.193120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:21.193442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:21.194374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.195433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:21.195433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.195433 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:21.219337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:21.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:21.219880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:21.220019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:21.220352 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:21.220493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.220938 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:21.221846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.222579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.223036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:21.223605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:21.224229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.225929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:21.285160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:21.285887 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:21.286231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:21.286231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:21.286987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:21.287415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.288418 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:21.289578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.290260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.290639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:21.291117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:21.291626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:21.292231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.292836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:21.303208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:21.303484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:21.303555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:21.303698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:21.304211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:21.304591 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:21.304848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.305608 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.306684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.307363 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:21.308113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.308113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:21.308316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:21.308993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:21.310159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:21.310159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:21.310159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:25.869258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:25.870066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:25.870066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:25.871299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:25.902541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:25.903039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:25.903921 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:25.904754 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:25.905418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:25.906260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:25.907011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:25.909227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:25.909227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:25.909227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:25.909227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:25.909227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:25.909227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:26.188217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:26.188749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:26.189028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:26.190197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:26.190644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:26.190967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.191904 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.193447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.194611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:26.195870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:26.196505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.197355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:26.197355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:26.197355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.197355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:26.197355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.197355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:26.219884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:26.220375 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:26.220555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:26.220739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:26.221337 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:26.221584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.222228 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:26.223479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.223993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.224969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:26.225328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:26.225686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.225686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:26.225686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.225686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:26.226640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.227120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.227574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:26.285880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:26.286366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:26.286664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:26.287090 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:26.288030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:26.288439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.289580 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:26.290951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.291748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.292200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:26.293097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:26.294159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:26.294946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.295805 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:26.303683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:26.303967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:26.304094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:26.304257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:26.305404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:26.305404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:26.305513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.306319 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.307621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.308022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:26.308430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.308430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:26.309391 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:26.309936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:26.310505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:26.310505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:26.310505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:30.869787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:30.870364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:30.870441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:30.871787 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:30.906650 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:30.906993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:30.907857 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:30.908565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:30.908995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:30.909347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:30.909825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:30.910662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:30.910662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:30.910662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:30.910662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:30.910662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:30.910662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:31.188802 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:31.189413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:31.190897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:31.190897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:31.191298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:31.191780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.193995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.194341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.194849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:31.195479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.195479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:31.196285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:31.196686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.197188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:31.197188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.197188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:31.220373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:31.220801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:31.221141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:31.221371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:31.222330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:31.222780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.224947 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:31.226264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.226961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:31.228385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:31.229280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.229280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:31.229280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.229280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:31.229650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.230355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.230948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:31.286428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:31.287078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:31.287278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:31.287520 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:31.288274 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:31.288633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.289479 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:31.290919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.291823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.292441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:31.293234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:31.294236 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:31.294978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.296604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:31.304187 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:31.304458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:31.304587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:31.304843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:31.305447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:31.305704 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:31.305876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.306547 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.308056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.308388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:31.308774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.308774 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:31.309751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:31.310283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:31.310802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:31.310802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:31.310802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:35.870661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:35.871555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:35.871555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:35.872609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:35.907956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:35.908131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:35.908518 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:35.909188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:35.909935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:35.910686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:35.911611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:35.912543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:35.912543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:35.912543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:35.912543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:35.912543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:35.912543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:36.189618 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:36.190167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:36.190260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:36.190824 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:36.190998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:36.191090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.191592 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.192305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.192888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:36.193408 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:36.194126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.194731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:36.194731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:36.194731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.194731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:36.194731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.194731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:36.221041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:36.221601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:36.221856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:36.222120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:36.222782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:36.223042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.223747 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:36.225181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.225648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.226732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:36.227239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:36.227631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.227631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:36.227631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.227631 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:36.228855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.229349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.229894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:36.287120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:36.287621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:36.287955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:36.288259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:36.289128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:36.289513 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.290502 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:36.291995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.292917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.293515 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:36.294353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:36.294960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.294960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:36.294960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.294960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.295677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:36.296410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:36.297122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.297122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:36.297122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.297122 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:36.304837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:36.305176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:36.305319 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:36.305541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:36.305988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:36.306294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:36.306455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.307064 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.308105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.308340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:36.308565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.308565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:36.309145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:36.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:36.309784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:36.309784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:36.309784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:40.871303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:40.872021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:40.872021 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:40.873507 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:40.910072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:40.910234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:40.910842 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:40.912091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:40.912788 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:40.913401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:40.915247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:40.916988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:40.916988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:40.916988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:40.916988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:40.916988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:40.916988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:41.190344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:41.190847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:41.191376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:41.191897 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:41.192367 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:41.192671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.194504 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.195295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.195880 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:41.196350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:41.197204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.197901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:41.197901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:41.197901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.197901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:41.197901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.197901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:41.221720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:41.222202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:41.222283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:41.222382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:41.223190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:41.223620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.227791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:41.228595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.229381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.229825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:41.230141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.230141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:41.231227 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.231616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.232011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:41.287959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:41.288428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:41.288628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:41.288973 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:41.289853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:41.290329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.291656 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:41.293127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.293658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.294644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:41.295079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:41.295491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.295491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:41.295491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.295491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.297268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:41.297900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:41.298439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.298439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:41.298439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.298439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:41.305242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:41.305965 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:41.306204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:41.306458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:41.306980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:41.307243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:41.307395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.308120 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.309800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.310242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:41.310586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.310586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:41.311550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:41.312029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:41.312523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:41.312523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:41.312523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:45.872037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:45.873121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:45.873121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:45.874477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:45.906834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:45.906834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:45.907228 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:45.908029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:45.908664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:45.909336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:45.910232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:45.911232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:45.911232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:45.911232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:45.911232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:45.911232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:45.911232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:46.191137 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:46.191679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:46.191889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:46.192955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:46.193355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:46.193619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.194470 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.195585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.195986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:46.196357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.196357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:46.196838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:46.197278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.197854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:46.197854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.197854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:46.222412 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:46.222829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:46.223000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:46.223159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:46.223686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:46.224127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.224958 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:46.225699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.226538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:46.226928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:46.227302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.227302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:46.227302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.227302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:46.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.228098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.228569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:46.288695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:46.289235 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:46.289431 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:46.289641 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:46.290247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:46.290554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.291456 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:46.292973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.293453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.294529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:46.294982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:46.295351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.295351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:46.295351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.295351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.296948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:46.297454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:46.297941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.297941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:46.297941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.297941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:46.306012 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:46.306440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:46.306624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:46.307001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:46.307860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:46.308389 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:46.308697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.309747 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.311324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.311665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:46.311998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.311998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:46.314323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:46.315001 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:46.315553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:46.315553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:46.315553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:50.872623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:50.873139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:50.873139 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:50.873497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:50.906119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:50.906468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:50.907404 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:50.908618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:50.909295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:50.910168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:50.911006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:50.911975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:50.911975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:50.911975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:50.911975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:50.911975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:50.911975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:51.191910 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:51.192228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:51.192316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:51.193638 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:51.194086 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:51.194086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.194869 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.195712 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.196443 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:51.197033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:51.197922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.198746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:51.198746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:51.198746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.198746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:51.198746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.198746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:51.223022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:51.223403 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:51.223693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:51.223947 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:51.224898 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:51.225173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.226297 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:51.227183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.227751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.228108 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:51.228549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:51.229052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.229052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:51.229052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.229052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:51.229188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.229678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.230357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:51.289471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:51.289967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:51.290290 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:51.290721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:51.291790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:51.292292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.293369 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:51.294481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:51.295706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:51.296351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.296962 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.298264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:51.299107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:51.299959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.299959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:51.299959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.299959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:51.306697 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:51.307059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:51.307193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:51.307383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:51.307862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:51.308117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:51.308286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.308937 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.310192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.310679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:51.311049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.311049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:51.311240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:51.311738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:51.312789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:51.312789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:51.312789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:55.873286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:55.874394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:55.874394 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:55.875624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:55.912899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:55.913225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:55.915776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:55.916176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:55.916531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:55.916919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:55.917723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:56.192497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:56.194125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:56.194125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:56.194736 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:56.195241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:56.195575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.196782 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.198458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.199177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:56.199824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.199824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:56.200732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:56.201555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.202614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:56.202614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.202614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:56.223637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:56.224177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:56.224378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:56.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:04:56.225152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:56.225369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.226031 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:56.227360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.227850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.228854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:56.229503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:56.230215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.230215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:56.230215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.230215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:56.230870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.231625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.232549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:56.290146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:56.290699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:56.290933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:56.291185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:56.292067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:56.292556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.293627 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:04:56.295006 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.295621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.296670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:56.297285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:56.297895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:56.298497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.299295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:04:56.307252 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:04:56.307584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:04:56.307747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:04:56.308041 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:04:56.308792 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:56.309288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:04:56.309524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.310348 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.311635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:04:56.311982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.311982 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:04:56.312403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:04:56.312942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:04:56.313853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:04:56.313853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:04:56.313853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:00.873895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:00.874832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:00.874832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:00.875020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:00.905173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:00.905575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:00.906624 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:00.909808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:00.910445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:00.911042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:00.911758 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:00.912676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:00.912676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:00.912676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:00.912676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:00.912676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:00.912676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:01.193217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:01.193715 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:01.193880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:01.194850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:01.195189 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:01.195431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.196390 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.197414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.198015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:01.198563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:01.199229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.200034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:01.200034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:01.200034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.200034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:01.200034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.200034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:01.224684 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:01.225108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:01.225215 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:01.225345 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:01.226014 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:01.226317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.227085 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:01.227959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.228634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.229447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:01.229836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:01.230248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.230248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:01.230248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.230248 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:01.232891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.233466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.233944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:01.290817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:01.291865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:01.291865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:01.291865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:01.292404 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:01.292614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.293199 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:01.294370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.295107 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.296290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:01.298104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:01.299496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:01.300099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.300888 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:01.307700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:01.308065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:01.308185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:01.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:01.308781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:01.309013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:01.309124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.309584 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.310454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.311032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:01.311524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:01.312148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:01.312902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:01.312902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:01.312902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:01.312902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:01.312902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:01.312902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:05.874461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:05.875271 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:05.875271 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:05.876586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:05.916461 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:05.916670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:05.917151 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:05.918141 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:05.918651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:05.919056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:05.919528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:05.920023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:05.920023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:05.920023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:05.920023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:05.920023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:05.920023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:06.195702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:06.196243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:06.196340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:06.197329 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:06.197670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:06.197973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.198859 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.199862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.200252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:06.200618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.200618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:06.201089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:06.201993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.203402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:06.203402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.203402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:06.225316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:06.225841 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:06.226038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:06.226285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:06.226862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:06.227175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.227990 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:06.230968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.231655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.232785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:06.233699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:06.234588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.235375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.235903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:06.291559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:06.291970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:06.292152 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:06.292372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:06.293244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:06.293664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.294883 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:06.296288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.296608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.297664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:06.298060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:06.298313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.298313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:06.298313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.298313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.299502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:06.299873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:06.300202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.300202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:06.300202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.300202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:06.308128 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:06.308355 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:06.308473 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:06.308624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:06.309084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:06.309343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:06.309488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.310659 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.311069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.311385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:06.311607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.311607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:06.312093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:06.312429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:06.312797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:06.312797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:06.312797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:10.875160 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:10.876053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:10.876053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:10.876246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:10.909642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:10.910106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:10.910963 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:10.911744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:10.912325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:10.912844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:10.913507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:10.914271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:10.914271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:10.914271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:10.914271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:10.914271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:10.914271 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:11.196411 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:11.196977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:11.197192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:11.198456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:11.198862 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:11.199117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.200193 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.201800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.202336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:11.202989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.202989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:11.203791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:11.204852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.205548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:11.205548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.205548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:11.225799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:11.226199 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:11.226387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:11.226579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:11.227353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:11.227688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.228673 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:11.230026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.231503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:11.232127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:11.233032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.233032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:11.233032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.233032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:11.233878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.234695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.235624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:11.292040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:11.292405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:11.292405 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:11.292546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:11.293245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:11.293538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.294175 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:11.295881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.296305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.298160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:11.298548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:11.298960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.298960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:11.298960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.298960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.300566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:11.301099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:11.301659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.301659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:11.301659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.301659 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:11.308479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:11.308729 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:11.308917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:11.309083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:11.309457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:11.309701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:11.309887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.310439 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.311468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.312009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:11.312386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.312386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:11.312683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:11.313230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:11.314095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:11.314095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:11.314095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:15.875740 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:15.876498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:15.876804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:15.878284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:15.919255 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:15.919775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:15.920894 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:15.921738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:15.922412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:15.923345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:15.924233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:15.925543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:15.925543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:15.925543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:15.925543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:15.925543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:15.925543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:16.198198 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:16.200986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:16.200986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:16.201588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:16.201588 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:16.201779 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.202410 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.203528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.203965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:16.204324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.204324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:16.204626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:16.205065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.205594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:16.205594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.205594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:16.226335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:16.226739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:16.226831 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:16.227088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:16.227541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:16.227929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.228673 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:16.229563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.230121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.230403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:16.230804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:16.231193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.231595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.232119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:16.292794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:16.293059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:16.293059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:16.293175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:16.293771 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:16.294037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.295014 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:16.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:16.297027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:16.298330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:16.299322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:16.300098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.301152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:16.308956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:16.309417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:16.309592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:16.309807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:16.310272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:16.310481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:16.310637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.311254 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.313150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.313633 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:16.314026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:16.314486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:16.315099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:16.315099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:16.315099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:16.315099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:16.315099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:16.315099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:20.876622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:20.877350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:20.877350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:20.878314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:20.885278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:05:20.915230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:20.915434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:20.916140 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:20.917345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:20.917864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:20.918187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:20.918187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:20.918547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:20.919023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:20.919488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:20.919488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:20.919488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:21.199997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:21.201017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:21.201017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:21.202146 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:21.202523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:21.202744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.203824 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.208328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.209019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:21.209579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:21.210345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:21.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:21.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:21.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:21.226791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:21.227281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:21.227468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:21.227667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:21.228351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:21.228870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.229483 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:21.230726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.231525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.232100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:21.232644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:21.233288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.233288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:21.233288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.233288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:21.233488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.234889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:21.293096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:21.293634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:21.293786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:21.293917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:21.294733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:21.295073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.295820 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:21.296802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.297540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.297967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:21.298457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:21.299217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.299217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:21.299217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.299217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.299469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:21.300720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:21.302409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.302409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:21.302409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.302409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:21.309444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:21.309967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:21.310197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:21.310459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:21.311293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:21.312106 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:21.312469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.313256 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.315053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.315516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:21.315903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.315903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:21.317050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:21.317527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:21.318624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:21.318624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:21.318624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:25.877318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:25.877902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:25.877985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:25.878753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:25.922429 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:25.922429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:25.922700 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:25.923861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:25.924154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:25.924431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:25.924431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:25.925043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:25.925392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:25.925807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:25.925807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:25.925807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:26.201010 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:26.201826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:26.202054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:26.203167 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:26.204193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:26.204193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.204471 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.206347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.206696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:26.207297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.207297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:26.208683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:26.209448 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.210287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:26.210287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.210287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:26.227633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:26.228289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:26.228482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:26.228682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:26.229900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:26.229900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.230516 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:26.232028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:26.233626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.234506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.235131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:26.235701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.236434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.237347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:26.294116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:26.294856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:26.295066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:26.295298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:26.296194 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:26.296870 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.298110 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:26.299708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.300734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.301385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:26.301963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:26.302511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.302511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:26.302511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.302511 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.303768 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:26.304280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:26.304808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.304808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:26.304808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.304808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:26.310070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:26.310642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:26.310832 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:26.311118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:26.311720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:26.312049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:26.312207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.312830 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.313898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.314333 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:26.314730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.314730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:26.315015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:26.315544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:26.316337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:26.316337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:26.316337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:30.300351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:30.301314 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:30.301388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:30.309780 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 05 02:05:30.309997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105368) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 05 02:05:30.310162 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:05:30.345399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, resource={'id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'name': 'public', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-05T01:30:22Z', 'updated_at': '2026-05-05T01:30:31Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.345399 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-fd02efee-ef31-4e95-abc3-f00ac4a17a72, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.345814 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 0b35bbe8-e654-4276-b862-7f8f7dafd01d (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.346907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.347306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.347775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.347775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.347775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.347775 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:05:30.377522 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-9c748568-b194-4190-82bf-d97c4b780883, resource={'id': '9c748568-b194-4190-82bf-d97c4b780883', 'name': 'admin_net', 'tenant_id': '908fbd2899e94f178724b7b1ab3536a4', 'project_id': '908fbd2899e94f178724b7b1ab3536a4', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2fadfbfc-6dbb-4597-9d15-008b9db68e74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:33:40Z', 'updated_at': '2026-05-05T01:33:41Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 38517}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.377878 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-9c748568-b194-4190-82bf-d97c4b780883, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.378286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-21469276-d42a-4a01-90f3-16a7473f8a9f, external_ids_update=None, columns={'tag': 38517}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.378434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.379154 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 9c748568-b194-4190-82bf-d97c4b780883 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.380118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.380607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.381278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.381278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.381278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.381278 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:05:30.412013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, resource={'id': 'c4593571-c367-4d2b-9f28-1c78cd90fd82', 'name': 'private', 'tenant_id': 'c76a7a59014f43b585637131d3cb4350', 'project_id': 'c76a7a59014f43b585637131d3cb4350', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8b89d3f9-f4c0-4b58-9e08-aceab0d061a4', 'd4de8ee6-ead8-4279-b730-c1debf1d8bb0'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:30:16Z', 'updated_at': '2026-05-05T01:30:19Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 18883}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.412495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c4593571-c367-4d2b-9f28-1c78cd90fd82, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.413095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-877f4443-ff15-42f2-a1ec-9bf8d7b91fff, external_ids_update=None, columns={'tag': 18883}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.413308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.414285 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource c4593571-c367-4d2b-9f28-1c78cd90fd82 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.415232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.415622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.416244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.416244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.416244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.416244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:05:30.445083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, resource={'id': '28e436bb-9153-4de9-91bf-b593e5bb4624', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'tenant_id': 'a90a4936446744ada1645ce1babcd14c', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['f99f07b3-a224-4ffc-a902-1ba3a9fbeb17'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-05T01:34:17Z', 'updated_at': '2026-05-05T01:34:17Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7145}, resource_type=networks, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.445500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-28e436bb-9153-4de9-91bf-b593e5bb4624, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.446120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ce0d3308-46ec-49fe-a430-1e4266f655e6, external_ids_update=None, columns={'tag': 7145}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.446281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.446928 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 28e436bb-9153-4de9-91bf-b593e5bb4624 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2331, in update_network
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.447889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.448477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.449136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.449136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.449136 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.449136 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:05:30.536230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, resource={'id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'project_id': 'a90a4936446744ada1645ce1babcd14c', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'enable_snat': True}, 'gw_port_id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-3mn09', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'external_fixed_ips': [{'ip_address': '172.24.5.167', 'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d'}, {'ip_address': '2001:db8::248', 'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 3, 'tenant_id': 'a90a4936446744ada1645ce1babcd14c'}, resource_type=routers, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-3mn09', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3808387c-1eb6-4d79-a30c-73923e69eb7b"')], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.538863 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.539620 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.539620 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.539620 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.539620 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.539620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Unable to update router bca3048b-ab22-440e-a9f8-31e6bea3944a. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource bca3048b-ab22-440e-a9f8-31e6bea3944a (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 05 02:05:30.539944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.540577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.541264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Fixing resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports) at create/update {{(pid=105368) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 05 02:05:30.657657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, resource={'id': '3808387c-1eb6-4d79-a30c-73923e69eb7b', 'name': '', 'network_id': '0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'project_id': '', 'mac_address': 'fa:16:3e:53:d5:75', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'bca3048b-ab22-440e-a9f8-31e6bea3944a', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d', 'ip_address': '172.24.5.167'}, {'subnet_id': 'f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'ip_address': '2001:db8::248'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171612', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-05T01:34:18Z', 'updated_at': '2026-05-05T01:34:20Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2e13052c-75f9-4e0e-8d6f-4552540e3e9d f8c08c34-3daf-44bc-b1e0-a4ded989d29d', 'neutron:network_name': 'neutron-0b35bbe8-e654-4276-b862-7f8f7dafd01d', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::248/64', '172.24.5.167/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3808387c-1eb6-4d79-a30c-73923e69eb7b, lrouter_port=lrp-3808387c-1eb6-4d79-a30c-73923e69eb7b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.657923 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Maintenance task: Failed to fix resource 3808387c-1eb6-4d79-a30c-73923e69eb7b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1975, in update_router_port
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.659372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.660424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.661188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.661188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.661188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.661188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.661188 np0000171612 neutron-ovn-maintenance-worker[105368]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_for_inconsistencies finished in 0.356 seconds
May 05 02:05:30.877895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:30.878458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:30.878625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:30.879950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:30.911999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:30.912365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.913224 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.914003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.914609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:30.915207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:30.915979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:30.916896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:30.916896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:30.916896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:30.916896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:30.916896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:30.916896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:31.201810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:31.202469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:31.202649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:31.203939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:31.204369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:31.204650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.206409 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.208498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:31.209424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:31.210488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.211851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:31.211851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:31.211851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.211851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:31.211851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.211851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:31.228439 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:31.228954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:31.229084 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:31.229243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:31.229778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:31.230177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.231091 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:31.232238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.233653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:31.234331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:31.235076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.235435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.236184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:31.295039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:31.295606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:31.295782 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:31.295995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:31.297256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:31.297705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.298635 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:31.299598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.300326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.301116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:31.301549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:31.302146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:31.302845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.304465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:31.310555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:31.310954 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:31.311204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:31.311536 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:31.312000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:31.312231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:31.312386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.313126 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.314616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.315119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:31.315460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.315460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:31.316173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:31.316804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:31.317611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:31.317611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:31.317611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:35.878528 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:35.879415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:35.879626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:35.881120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:35.924869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:35.925162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:35.925976 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:35.926877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:35.927473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:35.928142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:35.928916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:35.929778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:35.929778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:35.929778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:35.929778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:35.929778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:35.929778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:36.202605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:36.203107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:36.203233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:36.204197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:36.204508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:36.204508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.205851 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.206444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.206883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:36.207295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.207295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:36.207663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:36.208451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.209851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:36.209851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.209851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:36.229002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:36.229424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:36.229687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:36.229932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:36.231330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:36.231330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.232220 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:36.233413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.234353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.234853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:36.235532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:36.236199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.236199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:36.236199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.236199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:36.236943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.237893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.238990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:36.295807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:36.296184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:36.296341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:36.296585 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:36.299838 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:36.300165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.301068 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:36.303503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.304654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.305181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:36.305797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:36.306451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.306451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:36.306451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.306451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.307264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:36.308240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:36.309312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.309312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:36.309312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.309312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:36.310942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:36.311123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:36.311208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:36.311356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:36.312125 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:36.312517 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:36.312745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.313618 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.314267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.314553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:36.314844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.314844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:36.314992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:36.315317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:36.315705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:36.315705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:36.315705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:40.879284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:40.880295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:40.880295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:40.881682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:40.917721 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:40.918464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:40.919413 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:40.920221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:40.920875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:40.921539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:40.922334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:40.923257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:40.923257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:40.923257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:40.923257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:40.923257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:40.923257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:41.203269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:41.203871 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:41.204065 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:41.205205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:41.205596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:41.205897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.206917 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.208512 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.209154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:41.209700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.209700 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:41.210932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:41.211618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.212182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:41.212182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.212182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:41.229695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:41.230361 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:41.230544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:41.230735 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:41.231672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:41.232094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.233116 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:41.234415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.234999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.236197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:41.236609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:41.237045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.237045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:41.237045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.237045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:41.237963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.238452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:41.296354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:41.297224 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:41.297444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:41.297801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:41.298554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:41.299217 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.300936 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:41.301699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.302730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.303299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:41.303734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:41.304117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.304117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:41.304117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.304117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.304547 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:41.305215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:41.305818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.305818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:41.305818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.305818 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:41.311380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:41.311631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:41.311755 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:41.311952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:41.312437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:41.312742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:41.312968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.313605 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.314941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.315346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:41.315717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.315717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:41.316524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:41.317284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:41.318059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:41.318059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:41.318059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:45.879896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:45.880647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:45.880867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:45.882225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:45.918277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:45.918784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:45.919797 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:45.920492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:45.921054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:45.921638 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:45.922285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:45.923302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:45.923302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:45.923302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:45.923302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:45.923302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:45.923302 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:46.203821 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:46.204644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:46.204935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:46.205970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:46.206524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:46.206717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.209181 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.211387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.211686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:46.211984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.211984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:46.212753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:46.213104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.213498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:46.213498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.213498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:46.230366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:46.230749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:46.230899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:46.231037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:46.231599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:46.231889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.232619 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:46.233861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.234281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.235330 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:46.236025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:46.236640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.236640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:46.236640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.236640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:46.237791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.238598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.239481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:46.297245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:46.297700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:46.297856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:46.298005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:46.298991 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:46.299289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.300039 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:46.301342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.301786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.302925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:46.303275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:46.303647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.303647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:46.303647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.303647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.304741 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:46.305226 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:46.305826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.305826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:46.305826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.305826 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:46.311891 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:46.312173 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:46.312339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:46.312407 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:46.313401 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:46.313725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:46.313916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.314626 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.315268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.315593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:46.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:46.317082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:46.317508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:46.318002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:46.318002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:46.318002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:50.880683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:50.881645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:50.881645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:50.882372 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:50.918019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:50.918534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:50.919257 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:50.920042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:50.920711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:50.921402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:50.922478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:50.924064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:50.924064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:50.924064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:50.924064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:50.924064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:50.924064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:51.204668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:51.205385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:51.205600 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:51.206725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:51.207114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:51.207358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.208366 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.210003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.210384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:51.210715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.210715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:51.211646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:51.212098 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.212609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:51.212609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.212609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:51.231075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:51.231640 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:51.231883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:51.232149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:51.232937 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:51.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.234494 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:51.235687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.236237 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.237125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:51.237538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:51.237965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.237965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:51.237965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.237965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:51.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.239445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.240026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:51.297861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:51.298668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:51.298996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:51.299318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:51.300036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:51.300452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.301407 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:51.302874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.303279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.304270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:51.304636 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:51.305021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.305021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:51.305021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.305021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.305736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:51.306356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:51.306886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.306886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:51.306886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.306886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:51.312343 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:51.312639 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:51.312852 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:51.313062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:51.313521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:51.313935 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:51.314177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.314876 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.315782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.316149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:51.316473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.316473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:51.317319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:51.317828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:51.318340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:51.318340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:51.318340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:55.881228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:55.882071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:55.882261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:55.883482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:55.921496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:55.921963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:55.922896 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:55.923901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:55.924489 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:55.925034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:55.925650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:55.926418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:55.926418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:55.926418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:55.926418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:55.926418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:55.926418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:56.206229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:56.206794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:56.206856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:56.208249 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:56.208495 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:56.208677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.209618 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.211163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.211426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:56.211654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.211654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:56.212865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:56.213374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.213806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:56.213806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.213806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:56.232062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:56.232613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:56.232828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:56.233083 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:05:56.233885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:56.234214 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.235240 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:56.236552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.237142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.238400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:56.239434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:56.239743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.239743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:56.239743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.239743 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:56.240113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.240958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.241572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:56.299212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:56.299212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:56.299449 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:56.299629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:56.300422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:56.300934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.302004 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:05:56.303623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.304092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.305620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:56.305957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:56.306288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.306288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:56.306288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.306288 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.308049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:56.308381 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:56.308714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.308714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:56.308714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.308714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:05:56.312732 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:05:56.313140 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:05:56.313272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:05:56.313468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:05:56.314077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:56.314367 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:05:56.314587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.315287 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.316313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.317150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:05:56.317505 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:05:56.317898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:05:56.318398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:05:56.318398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:05:56.318398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:05:56.318398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:05:56.318398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:05:56.318398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:00.881745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:00.882424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:00.882521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:00.884554 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:00.917115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:00.917528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:00.918469 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:00.919154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:00.919835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:00.920465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:00.921290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:00.922389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:00.922389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:00.922389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:00.922389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:00.922389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:00.922389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:01.206158 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:01.206958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:01.206958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:01.207794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:01.208150 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:01.208379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.209186 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.210212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.210829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:01.211415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:01.212132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.213078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:01.213078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:01.213078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.213078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:01.213078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.213078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:01.232526 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:01.232955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:01.233174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:01.233371 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:01.234134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:01.234404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.235208 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:01.236474 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.236986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.237876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:01.238232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:01.238608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.238608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:01.238608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.238608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:01.239324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.239932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.240641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:01.299127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:01.299555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:01.299770 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:01.300018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:01.301062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:01.301350 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.302106 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:01.303994 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.304646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.305738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:01.306358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:01.306895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.306895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:01.306895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.306895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.307425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:01.307938 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:01.308536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.308536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:01.308536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.308536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:01.313093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:01.313348 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:01.313457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:01.313615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:01.314484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:01.314484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:01.314618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.315282 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.316418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.316988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:01.317375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.317375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:01.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:01.318907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:01.319466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:01.319466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:01.319466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:05.882288 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:05.883197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:05.883197 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:05.883579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:05.923967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:05.924251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:05.925024 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:05.925809 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:05.926499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:05.927201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:05.928440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:05.930679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:05.930679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:05.930679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:05.930679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:05.930679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:05.930679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:06.207093 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:06.208029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:06.208302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:06.209366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:06.209603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:06.209787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.210419 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.211478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.212144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:06.212816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:06.213708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.214747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:06.214747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:06.214747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.214747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:06.214747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.214747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:06.233277 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:06.233820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:06.233971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:06.234183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:06.234856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:06.235181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.236284 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:06.238013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.238868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.239492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:06.240219 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:06.240921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.240921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:06.240921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.240921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:06.241125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.242318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.243281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:06.300163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:06.300691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:06.300813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:06.301019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:06.301728 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:06.302050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.302775 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:06.304104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.304860 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.305539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:06.306127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:06.306825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.306825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:06.306825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.306825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.307850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:06.308442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:06.309165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.309165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:06.309165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.309165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:06.313783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:06.314213 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:06.314303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:06.314488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:06.315261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:06.315542 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:06.315729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.316575 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.318326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.319074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:06.319714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.319714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:06.319858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:06.320686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:06.321630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:06.321630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:06.321630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:10.882844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:10.911047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:10.911188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:10.912269 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:10.949800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:10.950183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:10.951109 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:10.952149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:10.952787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:10.953338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:10.954037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:10.955029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:10.955029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:10.955029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:10.955029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:10.955029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:10.955029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:11.207826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:11.208544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:11.208742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:11.209880 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:11.210245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:11.210587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.211409 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.212953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.213440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:11.213827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.213827 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:11.214645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:11.215084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.215574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:11.215574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.215574 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:11.233913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:11.234254 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:11.234437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:11.234747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:11.235257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:11.235614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.236563 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:11.237780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.238540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.239106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:11.239728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:11.240452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.240452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:11.240452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.240452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:11.240736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.241303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.241816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:11.300836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:11.301386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:11.301613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:11.301865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:11.302593 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:11.302975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.303912 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:11.308942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.309729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.310346 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:11.310917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:11.311326 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:11.311717 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.312239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:11.314256 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:11.314488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:11.314568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:11.314691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:11.315168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:11.315514 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:11.315708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.316403 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.317077 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.317425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:11.317793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:11.318242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:11.318857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:11.318857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:11.318857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:11.318857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:11.318857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:11.318857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:15.911185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:15.911913 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:15.912116 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:15.913415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:15.947907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:15.948447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:15.949354 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:15.950140 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:15.950575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:15.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:15.951747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:15.952417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:15.952417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:15.952417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:15.952417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:15.952417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:15.952417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:16.208668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:16.209242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:16.209467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:16.210631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:16.211006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:16.211304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.212414 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.214256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.214977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:16.215650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.215650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:16.216270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:16.217075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.218029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:16.218029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.218029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:16.234410 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:16.234920 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:16.235147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:16.235402 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:16.236062 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:16.236370 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.237179 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:16.238447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.239347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.239931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:16.240622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:16.241315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.242046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:16.301645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:16.302253 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:16.302464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:16.302695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:16.303647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:16.304148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.305052 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:16.306533 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.306953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.308374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:16.308721 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:16.309103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.309103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:16.309103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.309103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.310329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:16.311042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:16.311822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.311822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:16.311822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.311822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:16.314738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:16.315085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:16.315211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:16.315374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:16.315899 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:16.316376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:16.316637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.317407 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.318036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.318495 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:16.319147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:16.319901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:16.320858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:16.320858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:16.320858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:16.320858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:16.320858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:16.320858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:20.911924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:20.912701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:20.912701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:20.916044 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:20.918020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=433 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:06:20.949534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:20.949914 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:20.950600 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:20.951284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:20.951977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:20.952480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:20.953353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:20.954143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:20.954143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:20.954143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:20.954143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:20.954143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:20.954143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:21.209259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:21.209956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:21.210231 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:21.211457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:21.211730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:21.211878 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.212611 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.214259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.216965 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:21.217521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:21.218005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.218581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:21.218581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:21.218581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.218581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:21.218581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.218581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:21.234907 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:21.235387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:21.235636 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:21.235900 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:21.236727 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:21.236924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.237594 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:21.238967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.239556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.240344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:21.240807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:21.241239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.241239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:21.241239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.241239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:21.242037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.242530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.243051 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:21.302497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:21.302904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:21.303022 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:21.303203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:21.303931 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:21.304403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.305349 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:21.306420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:21.308460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:21.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:21.309670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:21.310044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.310571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:21.315245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:21.315559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:21.315719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:21.315922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:21.316438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:21.316918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:21.317166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.318014 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.319484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.319904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:21.320280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.320280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:21.321296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:21.321838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:21.322373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:21.322373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:21.322373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:25.912532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:25.913298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:25.913524 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:25.915302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:25.948934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:25.949269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:25.949643 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:25.950332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:25.951052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:25.951651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:25.952154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:25.952685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:25.952685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:25.952685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:25.952685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:25.952685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:25.952685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:26.209872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:26.210471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:26.210670 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:26.211745 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:26.211979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:26.212148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.212873 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.213845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.214361 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:26.214843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.214843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:26.214999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:26.215587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.216299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:26.216299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.216299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:26.235382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:26.235953 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:26.236156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:26.236336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:26.236938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:26.237239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.238130 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:26.239436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.240518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.241184 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:26.242026 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:26.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.243731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.244467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:26.303095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:26.303720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:26.303950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:26.304172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:26.305117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:26.305540 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.306570 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:26.309424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.309867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.311084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:26.311412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:26.311865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.311865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:26.311865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.311865 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.312970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:26.313429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:26.313921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.313921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:26.313921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.313921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:26.315552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:26.315718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:26.315858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:26.315977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:26.316505 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:26.317006 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:26.317282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.318090 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.318771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.319104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:26.319418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.319418 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:26.319541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:26.319964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:26.320411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:26.320411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:26.320411 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:30.913308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:30.914193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:30.914193 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:30.915475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:30.956518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:30.956943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:30.958053 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:30.958950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:30.959569 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:30.960176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:30.960898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:30.961740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:30.961740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:30.961740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:30.961740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:30.961740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:30.961740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:31.210651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:31.211195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:31.211400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:31.212304 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:31.212488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:31.212657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.213874 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.215469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.216088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:31.216588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.216588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:31.217955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:31.218656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.219528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:31.219528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.219528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:31.235993 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:31.236481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:31.236691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:31.236940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:31.237494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:31.237795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.238541 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:31.239696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.240181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.240947 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:31.241395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:31.241863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.241863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:31.241863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.241863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:31.242966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.243643 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.244384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:31.303925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:31.304521 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:31.304742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:31.304985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:31.305537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:31.306437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.307283 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:31.308522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.308987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.309977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:31.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:31.310884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.310884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:31.310884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.310884 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.312015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:31.312481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:31.313943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.313943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:31.313943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.313943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:31.316049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:31.316367 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:31.316450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:31.316634 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:31.316980 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:31.317216 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:31.317348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.317795 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.318582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.319523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:31.320476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:31.321283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:31.322354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:31.322354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:31.322354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:31.322354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:31.322354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:31.322354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:35.914121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:35.915027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:35.915181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:35.916688 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:35.958482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:35.958778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:35.961055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:35.961423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:35.961719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:35.961719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:35.961719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:35.961719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:35.961719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:35.961719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:35.962472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:35.962825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:35.963181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:35.963181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:35.963181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:36.211237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:36.211559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:36.211784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:36.213195 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:36.213626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:36.214023 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.214895 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.216536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.217703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:36.218634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:36.219734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.221156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:36.221156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:36.221156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.221156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:36.221156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.221156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:36.236668 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:36.237147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:36.237415 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:36.237781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:36.238438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:36.238787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.239597 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:36.240405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.241104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.241522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:36.242066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:36.242568 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.243153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.243927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:36.304642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:36.305163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:36.305417 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:36.305809 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:36.306691 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:36.307396 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.308821 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:36.309728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.310491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.310928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:36.311514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:36.312179 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:36.312926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.313480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:36.316434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:36.316738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:36.316989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:36.317228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:36.317649 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:36.318108 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:36.318223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.318932 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.320181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.320651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:36.321048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.321048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:36.321684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:36.322190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:36.322727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:36.322727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:36.322727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:40.914730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:40.915906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:40.915906 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:40.917362 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:40.954239 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:40.954470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:40.955135 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:40.956297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:40.956685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:40.957109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:40.957109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:40.957384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:40.957722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:40.958295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:40.958295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:40.958295 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:41.211675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:41.212186 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:41.212308 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:41.213121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:41.213390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:41.213630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.214506 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.216117 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.216379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:41.216609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.216609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:41.217250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:41.217556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.217926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:41.217926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.217926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:41.237311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:41.237642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:41.237804 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:41.237889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:41.238579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:41.239017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.239891 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:41.240627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.241055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.241486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:41.241838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:41.242212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.242212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:41.242212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.242212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:41.242579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.243009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.243519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:41.305161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:41.305519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:41.305952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:41.306456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:41.307151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:41.307509 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.308398 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:41.309736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.310404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.311241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:41.311571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:41.312149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.312149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:41.312149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.312149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.312798 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:41.313304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:41.313890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.313890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:41.313890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.313890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:41.317583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:41.317820 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:41.317927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:41.318097 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:41.318472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:41.318751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:41.318963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.319531 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.320274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.320742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:41.321094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:41.321531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:41.322049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:41.322049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:41.322049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:41.322049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:41.322049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:41.322049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:45.915525 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:45.916406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:45.916406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:45.917138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:45.958949 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:45.959160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:45.959856 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:45.961010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:45.961493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:45.962099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:45.962099 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:45.962639 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:45.963180 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:45.963863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:45.963863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:45.963863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:46.212378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:46.212626 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:46.212836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:46.213692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:46.214069 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:46.214297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.215070 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.215856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.216437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:46.217181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:46.217887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.218869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:46.218869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:46.218869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.218869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:46.218869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.218869 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:46.237807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:46.238171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:46.238414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:46.238633 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:46.239312 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:46.239545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.240286 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:46.241705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.242896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.243611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:46.244441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:46.245260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.245260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:46.245260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.245260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:46.246165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.246653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.247045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:46.305724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:46.306038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:46.306144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:46.306294 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:46.307082 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:46.307492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.308344 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:46.309378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.310177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.310602 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:46.311175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:46.311858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:46.312752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.314949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:46.318011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:46.318774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:46.319272 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:46.319652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:46.319930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:46.320078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.320734 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.322375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.322710 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:46.322992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.322992 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:46.324177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:46.324550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:46.325100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:46.325100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:46.325100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:50.916560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:50.917465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:50.917465 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:50.919996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:50.956512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:50.957310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:50.958272 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:50.960033 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:50.960460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:50.960882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:50.960882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:50.961163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:50.961673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:50.962266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:50.962266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:50.962266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:51.213114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:51.213632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:51.213859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:51.215094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:51.215373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:51.215534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.216999 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.218698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.219269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:51.219832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.219832 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:51.220434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:51.221039 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.221605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:51.221605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.221605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:51.238386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:51.239183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:51.239413 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:51.239616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:51.240366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:51.240903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.241891 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:51.243327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.244046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.244680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:51.245206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:51.245644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.245644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:51.245644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.245644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:51.246425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.246898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.247440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:51.306459 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:51.306867 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:51.306989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:51.307115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:51.307747 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:51.308188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.309019 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:51.310458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:51.311127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:51.312138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:51.312836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:51.313432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.314479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:51.318865 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:51.319100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:51.319218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:51.319388 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:51.319883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:51.320263 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:51.320510 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.321238 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.322102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.322560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:51.322972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.322972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:51.323773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:51.324257 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:51.324748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:51.324748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:51.324748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:55.917092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:55.917719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:55.917800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:55.918749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:55.953476 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:55.953701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:55.954435 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:55.955234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:55.955523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:55.955777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:55.955777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:55.956082 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:55.956508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:55.956892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:55.956892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:55.956892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:56.214036 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:56.214477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:56.214606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:56.215719 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:56.216169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:56.216380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.217297 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.218008 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.218605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:56.219203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:56.219952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.220811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:56.220811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:56.220811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.220811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:56.220811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.220811 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:56.239018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:56.239453 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:56.239657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:56.239930 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:06:56.240621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:56.241054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.241930 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:56.243290 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.244105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.244750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:56.245472 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:56.246444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.246444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:56.246444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.246444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:56.246908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.247603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.248614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:56.306986 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:56.307595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:56.307889 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:56.308181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:56.308885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:56.309283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.310204 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:06:56.311932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.312496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.313781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:56.314119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:56.314476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.314476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:56.314476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.314476 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.315967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:56.316514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:56.317131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.317131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:56.317131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.317131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:06:56.319297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:06:56.319733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:06:56.319733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:06:56.319924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:06:56.320247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:56.320484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:06:56.320704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.321123 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.323259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.323683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:06:56.324052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.324052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:06:56.324676 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:06:56.325190 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:06:56.325746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:06:56.325746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:06:56.325746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:00.917868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:00.918807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:00.918807 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:00.920029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:00.956184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:00.956576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:00.957578 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:00.958686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:00.959364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:00.960068 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:00.960897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:00.961872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:00.961872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:00.961872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:00.961872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:00.961872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:00.961872 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:01.214711 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:01.215172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:01.215259 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:01.216558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:01.217035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:01.217402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.218567 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.220279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.220702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:01.221094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.221094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:01.222196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:01.222681 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.223289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:01.223289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.223289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:01.239676 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:01.240138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:01.240317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:01.240510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:01.240959 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:01.241194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.241807 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:01.242989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.243735 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.244276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:01.244894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:01.245521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.245521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:01.245521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.245521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:01.246043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.246858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.247623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:01.307589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:01.307999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:01.308219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:01.308541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:01.309037 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:01.309300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.310127 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:01.311912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.312609 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.313484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:01.314188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:01.314934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.314934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:01.314934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.314934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.315950 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:01.316871 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:01.318289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.318289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:01.318289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.318289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:01.319928 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:01.320229 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:01.320358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:01.320541 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:01.321016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:01.321369 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:01.321655 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.322405 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.323390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.323833 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:01.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:01.324479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:01.324985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:01.325562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:01.325562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:01.325562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:05.919258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:05.920144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:05.920190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:05.921165 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:05.956872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:05.957292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:05.958348 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:05.959519 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:05.959941 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:05.960364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:05.960364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:05.960789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:05.961336 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:05.961986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:05.961986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:05.961986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:06.219385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:06.220568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:06.220744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:06.221625 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:06.221998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:06.222241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.223114 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.224188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.224625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:06.225043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.225043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:06.225340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:06.225858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.226452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:06.226452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.226452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:06.240245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:06.240583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:06.240776 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:06.241016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:06.241398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:06.241561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.242131 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:06.243558 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.244883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.245618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:06.246407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:06.247149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.247149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:06.247149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.247149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:06.247535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.248316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.249247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:06.308330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:06.308904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:06.309163 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:06.309446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:06.310109 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:06.310385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.311080 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:06.312242 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.312825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:06.313705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:06.314132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.314132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:06.314132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.314132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.314698 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:06.315303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:06.315922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.315922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:06.315922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.315922 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:06.320265 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:06.320550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:06.320714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:06.320982 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:06.321458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:06.321872 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:06.322118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.323115 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.325010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:06.325373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.325373 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:06.325695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:06.326151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:06.326664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:06.326664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:06.326664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:10.920331 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:10.921107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:10.921107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:10.921660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:10.958386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:10.958645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:10.958975 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:10.959699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:10.960069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:10.960612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:10.961080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:10.961536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:10.961536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:10.961536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:10.961536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:10.961536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:10.961536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:11.220577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:11.221300 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:11.221608 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:11.223222 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:11.223819 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:11.224230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.225135 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.226742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.227143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:11.227455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.227455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:11.228406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:11.228916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.229379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:11.229379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.229379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:11.240642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:11.261621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:11.261621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:11.261621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:11.262085 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:11.262321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.264120 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:11.265591 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.266032 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.267244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:11.267612 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:11.268036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.268036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:11.268036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.268036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:11.269451 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.269988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.270430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:11.308940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:11.309297 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:11.309441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:11.309594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:11.310353 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:11.310749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.311650 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:11.312635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.313048 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.313666 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:11.313981 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:11.314312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.314312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:11.314312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.314312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.314777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:11.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:11.315679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.315679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:11.315679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.315679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:11.320624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:11.321370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:11.321370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:11.321370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:11.321582 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:11.321914 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:11.322125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:11.324313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:11.324740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.325144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:11.325144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:11.325144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.325144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:11.325144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.325144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:11.325334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:11.325738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:11.326206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:11.326206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:11.326206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:15.921038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:15.921958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:15.921958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:15.923144 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:15.955289 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:15.955685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:15.956563 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:15.957488 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:15.958215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:15.958964 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:15.959836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:15.960782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:15.960782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:15.960782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:15.960782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:15.960782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:15.960782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:16.221243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:16.221651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:16.221874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:16.223148 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:16.223511 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:16.223744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.224715 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.226590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.226923 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:16.227246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.227246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:16.229065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:16.229387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.229739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:16.229739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.229739 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:16.261322 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:16.261893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:16.262094 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:16.262315 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:16.262917 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:16.263172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.264651 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:16.266555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:16.268078 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.269291 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.270238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:16.271218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.272173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.273225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:16.309462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:16.309967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:16.310089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:16.310202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:16.310716 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:16.311005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.311562 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:16.312323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.312998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.313415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:16.313977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:16.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:16.315421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.316299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:16.320988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:16.321221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:16.321335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:16.321490 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:16.321885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:16.322172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:16.322348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.322835 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.324195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.324528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:16.324892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.324892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:16.325881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:16.326354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:16.326875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:16.326875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:16.326875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:20.921883 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:20.922664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:20.922664 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:20.923788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:20.925743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:07:20.962589 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:20.963000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:20.964270 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:20.965861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:20.966497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:20.967079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:20.967079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:20.967527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:20.968279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:20.969102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:20.969102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:20.969102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:21.221855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:21.222230 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:21.222424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:21.223577 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:21.224089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:21.224375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.225377 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.227100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.227996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:21.228581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.228581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:21.228691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:21.229441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.230325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:21.230325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.230325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:21.261921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:21.262299 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:21.262480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:21.262675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:21.263350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:21.263661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.264469 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:21.265653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.266470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.266983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:21.267557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:21.268178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.268178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:21.268178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.268178 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:21.268616 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.269153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.269843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:21.309853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:21.310344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:21.310558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:21.310753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:21.311095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:21.311337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.311825 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:21.312570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.313137 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.313487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:21.313874 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:21.314507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.314507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:21.314507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.314507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.315017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:21.315564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:21.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:21.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.316168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:21.321373 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:21.321621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:21.321733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:21.321927 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:21.322301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:21.322678 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:21.322945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.324281 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.325458 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.326024 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:21.326928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.326928 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:21.328621 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:21.329186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:21.329988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:21.329988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:21.329988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:25.922494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:25.924104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:25.924104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:25.924661 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:25.955673 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:25.956129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:25.957165 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:25.958182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:25.958622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:25.959031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:25.959031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:25.959307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:25.959804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:25.960337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:25.960337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:25.960337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:26.222498 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:26.223103 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:26.223311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:26.224338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:26.224663 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:26.224939 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.225866 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.227423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.228019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:26.228528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.228528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:26.229129 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:26.229929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.230780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:26.230780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.230780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:26.262478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:26.262961 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:26.263155 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:26.263365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:26.263971 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:26.264354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.265254 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:26.266535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.267470 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.267991 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:26.268592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:26.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:26.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.269276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:26.269929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.270800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.271617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:26.310363 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:26.310799 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:26.311011 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:26.311237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:26.311700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:26.311999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.312680 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:26.313988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.314580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:26.315916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:26.316351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.316351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:26.316351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.316351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.317103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:26.317625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:26.318118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.318118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:26.318118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.318118 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:26.321851 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:26.322202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:26.322317 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:26.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:26.322957 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:26.323378 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:26.323630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.324380 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.325216 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.325619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:26.325998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.325998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:26.326272 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:26.326773 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:26.327309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:26.327309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:26.327309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:30.923219 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:30.923893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:30.923893 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:30.925174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:30.965214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:30.966608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:30.966866 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:30.968091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:30.968737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:30.970034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:30.970828 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:30.972300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:30.972300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:30.972300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:30.972300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:30.972300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:30.972300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:31.222919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:31.223286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:31.223427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:31.225689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:31.225689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:31.225689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.225954 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.227044 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.227694 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:31.228110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.228110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:31.228177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:31.228695 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.229245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:31.229245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.229245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:31.262932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:31.263344 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:31.263530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:31.263800 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:31.265042 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:31.265042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.265359 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:31.266784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.267508 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.268173 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:31.268706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:31.269323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.269323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:31.269323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.269323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:31.270075 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.271128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.271910 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:31.310836 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:31.311246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:31.311434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:31.311682 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:31.312177 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:31.312546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.313303 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:31.314531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.315202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.316139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:31.316699 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:31.317332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.317332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:31.317332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.317332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.318225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:31.319016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:31.319847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.319847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:31.319847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.319847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:31.322182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:31.322509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:31.322615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:31.322781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:31.323258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:31.323622 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:31.323894 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.324737 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.325581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.325925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:31.326281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.326281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:31.326597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:31.327063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:31.327585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:31.327585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:31.327585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:35.923850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:35.924460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:35.924924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:35.926196 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:35.964400 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:35.966970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:35.968480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:35.969879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:35.970892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:35.972281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:36.223683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:36.224279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:36.224564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:36.225863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:36.226286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:36.226598 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.227778 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.229590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.230497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:36.231414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.231414 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:36.231997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:36.232879 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.233371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:36.233371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.233371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:36.263516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:36.264007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:36.264260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:36.264540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:36.265325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:36.265679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.266551 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:36.267690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.268407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.268942 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:36.269484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:36.270090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.270090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:36.270090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.270090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:36.270425 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.270944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.271419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:36.311357 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:36.311698 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:36.311859 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:36.312070 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:36.312874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:36.313318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.314139 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:36.315052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.315629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.316279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:36.316822 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:36.317369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.317369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:36.317369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.317369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.317844 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:36.318345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:36.318845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.318845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:36.318845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.318845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:36.322583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:36.322885 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:36.323030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:36.323241 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:36.323714 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:36.324153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:36.324464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.325344 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.326298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.326864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:36.327441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.327441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:36.327674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:36.328183 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:36.328696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:36.328696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:36.328696 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:40.924985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:40.926892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:40.926892 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:40.927171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:40.965039 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:40.965247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:40.966114 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:40.967192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:40.968031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:40.968648 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:40.970081 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:40.971320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:40.971320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:40.971320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:40.971320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:40.971320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:40.971320 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:41.224358 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:41.224940 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:41.225156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:41.226268 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:41.226703 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:41.227106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.228133 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.229692 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.230041 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:41.230407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.230407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:41.231790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:41.232243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.232813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:41.232813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.232813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:41.264089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:41.264592 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:41.264723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:41.264934 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:41.265683 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:41.266066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.267119 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:41.268067 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.268974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.269382 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:41.270144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:41.271086 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.272093 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.273430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:41.311922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:41.312502 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:41.312695 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:41.313000 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:41.313644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:41.313985 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.314707 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:41.316181 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.316594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.318057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:41.318389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:41.318836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.318836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:41.318836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.318836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.320013 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:41.320475 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:41.321142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.321142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:41.321142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.321142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:41.323001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:41.323295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:41.323458 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:41.323657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:41.324059 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:41.324335 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:41.324531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.325209 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.326380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.326813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:41.327166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.327166 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:41.327771 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:41.328213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:41.328709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:41.328709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:41.328709 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:45.925651 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:45.926374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:45.926629 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:45.927992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:45.967120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:45.967120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:45.967876 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:45.968664 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:45.969627 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:45.970625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:45.971945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:45.973281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:45.973281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:45.973281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:45.973281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:45.973281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:45.973281 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:46.225031 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:46.225571 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:46.225842 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:46.226884 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:46.227054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:46.227186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.227702 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.228957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.229581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:46.230163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.230163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:46.230378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:46.231148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.231669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:46.231669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.231669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:46.264794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:46.265422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:46.265631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:46.265830 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:46.266482 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:46.266911 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.267789 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:46.268703 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.269705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.270260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:46.271055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:46.271724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.272467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.273521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:46.312462 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:46.312925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:46.313127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:46.313340 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:46.314107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:46.314608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.315684 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:46.316975 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.317420 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.318620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:46.318997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:46.319343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.319343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:46.319343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.319343 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.320665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:46.321243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:46.321714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.321714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:46.321714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.321714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:46.323595 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:46.323916 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:46.324028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:46.324181 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:46.324523 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:46.324779 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:46.324989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.325506 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.326377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.326730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:46.327056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.327056 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:46.327379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:46.327847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:46.328348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:46.328348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:46.328348 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:50.926385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:50.927414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:50.927414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:50.928869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:50.967645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:50.968090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:50.969097 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:50.970171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:50.970546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:50.970932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:50.970932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:50.971240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:50.971667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:50.972193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:50.972193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:50.972193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:51.225665 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:51.226242 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:51.226364 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:51.227623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:51.228076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:51.228318 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.229308 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.230993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.231618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:51.232364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.232364 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:51.234194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:51.234927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.235843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:51.235843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.235843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:51.265435 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:51.265938 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:51.266172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:51.266324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:51.266977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:51.267284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.268156 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:51.269497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.270514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.271171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:51.271953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:51.272857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.273838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.274389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:51.313067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:51.313644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:51.313855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:51.314080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:51.314692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:51.315143 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.316029 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:51.317399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.318261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.318955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:51.319566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:51.320233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.320233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:51.320233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.320233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.320656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:51.321463 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:51.322323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.322323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:51.322323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.322323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:51.324131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:51.324492 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:51.324675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:51.324983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:51.325422 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:51.325805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:51.326036 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.326656 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.327646 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.328349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:51.328856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.328856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:51.329015 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:51.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:51.330222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:51.330222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:51.330222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:55.927145 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:55.928075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:55.928075 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:55.929264 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:55.963801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:55.964027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:55.964647 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:55.965937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:55.966585 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:55.967251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:55.967251 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:55.967530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:55.969187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:55.970583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:55.970583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:55.970583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:56.226387 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:56.226791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:56.227024 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:56.228437 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:56.228709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:56.228937 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.229906 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.231559 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.232341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:56.232924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.232924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:56.233689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:56.234340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.235152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:56.235152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.235152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:56.266066 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:56.266496 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:56.266788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:56.267117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:07:56.267956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:56.268372 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.269608 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:56.271175 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.272057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.272921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:56.274085 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:56.274715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.275972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.276527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:56.313817 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:56.314316 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:56.314573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:56.314790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:56.315619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:56.316245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.317316 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:07:56.318958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.319514 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.320468 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:56.320969 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:56.321560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.321560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:56.321560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.321560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.321797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:56.322571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:56.323640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.323640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:56.323640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.323640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:07:56.324590 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:07:56.324924 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:07:56.325056 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:07:56.325201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:07:56.325788 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:56.326217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:07:56.326457 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.327232 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.328204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.328604 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:07:56.329031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.329031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:07:56.329278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:07:56.329751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:07:56.330387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:07:56.330387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:07:56.330387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:00.927686 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:00.928460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:00.928460 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:00.929904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:00.966129 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:00.966596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:00.968640 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:00.969617 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:00.970331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:00.970890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:00.970890 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:00.971194 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:00.971986 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:00.972850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:00.972850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:00.972850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:01.226874 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:01.227334 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:01.227556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:01.228356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:01.228572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:01.228701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.229304 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.230912 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.231520 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:01.232050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.232050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:01.232776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:01.233356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.234002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:01.234002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.234002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:01.266576 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:01.267214 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:01.267516 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:01.267774 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:01.268447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:01.268797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.269719 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:01.271255 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.272284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.272891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:01.273690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:01.274335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.274335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:01.274335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.274335 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:01.274625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.275265 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.275803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:01.314426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:01.314981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:01.315166 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:01.315376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:01.316095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:01.316629 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.317661 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:01.319053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.319570 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.320454 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:01.320836 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:01.321273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.321273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:01.321273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.321273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.322339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:01.322905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:01.323484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.323484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:01.323484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.323484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:01.325029 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:01.325326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:01.325420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:01.325615 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:01.326076 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:01.326433 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:01.326654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.327369 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.328269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.328619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:01.328984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.328984 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:01.329415 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:01.330121 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:01.331095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:01.331095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:01.331095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:05.928366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:05.929587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:05.929587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:05.930662 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:05.969117 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:05.969490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:05.970337 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:05.971229 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:05.972261 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:05.973460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:05.974803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:05.976264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:05.976264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:05.976264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:05.976264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:05.976264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:05.976264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:06.227527 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:06.227999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:06.228227 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:06.229310 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:06.229722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:06.230072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.231341 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.232783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.233417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:06.234072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.234072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:06.234796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:06.235583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.236674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:06.236674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.236674 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:06.267351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:06.267932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:06.268169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:06.268434 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:06.269102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:06.269430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.270301 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:06.271297 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.272120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.272549 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:06.273145 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:06.273927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.273927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:06.273927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.273927 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:06.274119 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.275092 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.276103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:06.314948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:06.315245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:06.315382 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:06.315539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:06.315999 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:06.316280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.316864 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:06.317834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.318412 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.319071 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:06.319622 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:06.320182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.320182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:06.320182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.320182 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.320782 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:06.321299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:06.321813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.321813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:06.321813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.321813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:06.325573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:06.325972 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:06.326159 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:06.326346 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:06.326713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:06.327050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:06.327239 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.327905 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.328711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.329230 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:06.329706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.329706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:06.329814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:06.330413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:06.331146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:06.331146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:06.331146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:10.928998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:10.929843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:10.929843 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:10.931234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:10.970983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:10.971441 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:10.972539 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:10.973432 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:10.974745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:10.975130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:10.975546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:10.976040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:10.976040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:10.976040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:10.976040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:10.976040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:10.976040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:11.228303 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:11.228743 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:11.228989 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:11.230233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:11.230573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:11.230823 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.231650 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.233225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.233892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:11.234503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.234503 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:11.234959 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:11.235723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.236589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:11.236589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.236589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:11.268013 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:11.268414 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:11.268605 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:11.268785 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:11.269637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:11.270055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.271201 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:11.272392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.273188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.273796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:11.274378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:11.275046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.275046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:11.275046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.275046 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:11.275560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.277317 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.278233 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:11.315447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:11.315905 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:11.316092 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:11.316291 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:11.317007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:11.317355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.318142 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:11.319347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.319943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.320784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:11.321152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:11.321634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.321634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:11.321634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.321634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.322577 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:11.323287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:11.324065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.324065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:11.324065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.324065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:11.326121 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:11.326436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:11.326555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:11.326710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:11.327133 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:11.327397 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:11.327561 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.328133 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.328971 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.329542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:11.330150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:11.330907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:11.331747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:11.331747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:11.331747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:11.331747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:11.331747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:11.331747 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:15.929677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:15.930468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:15.930596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:15.931559 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:15.970115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:15.970482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:15.971193 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:15.972058 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:15.972849 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:15.973653 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:15.974582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:15.975701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:15.975701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:15.975701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:15.975701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:15.975701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:15.975701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:16.229004 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:16.229391 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:16.229586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:16.230569 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:16.230902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:16.231168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.232009 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.233430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.233825 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:16.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.234163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:16.235220 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:16.235727 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.236244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:16.236244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.236244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:16.268472 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:16.268861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:16.269047 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:16.269240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:16.269856 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:16.270152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.270957 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:16.272308 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.273665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:16.274332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:16.275138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.275138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:16.275138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.275138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:16.275556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.276465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.277342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:16.316151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:16.316631 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:16.316725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:16.316919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:16.317578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:16.318405 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.319830 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:16.320682 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.321374 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.321940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:16.322522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:16.323221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.323221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:16.323221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.323221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.323542 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:16.324338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:16.325205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.325205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:16.325205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.325205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:16.326480 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:16.326702 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:16.326868 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:16.327078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:16.327598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:16.328179 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:16.328465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.329191 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.330247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.330669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:16.331154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.331154 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:16.331378 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:16.331876 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:16.332586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:16.332586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:16.332586 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:20.930750 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:20.931693 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:20.931850 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:20.933141 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:20.935081 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:08:20.974985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:20.975176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:20.975554 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:20.976328 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:20.977197 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:20.977968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:20.978900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:20.980016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:20.980016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:20.980016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:20.980016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:20.980016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:20.980016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:21.229922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:21.230584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:21.230881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:21.232080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:21.232483 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:21.232736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.233902 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.235060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.235658 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:21.236185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.236185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:21.236282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:21.237028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.237830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:21.237830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.237830 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:21.269282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:21.269943 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:21.270244 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:21.270533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:21.271815 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:21.272169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.273149 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:21.274342 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:21.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.276644 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.277486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:21.277968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.278607 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.279641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:21.317071 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:21.317667 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:21.317875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:21.318184 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:21.319182 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:21.319557 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.320497 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:21.322199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.323341 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.324110 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:21.324605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:21.325250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.325250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:21.325250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.325250 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.326260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:21.327898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:21.328713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:21.329149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:21.329149 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:21.329149 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.329521 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.330339 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:21.332238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:21.333218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:21.334389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:21.334389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:21.334389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:21.334389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:21.334389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:21.334389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:25.931563 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:25.932441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:25.932441 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:25.933545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:25.965539 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:25.966007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:25.966953 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:25.968088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:25.968527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:25.969012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:25.969012 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:25.969169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:25.969867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:25.970814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:25.970814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:25.970814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:26.231202 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:26.231791 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:26.231995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:26.233221 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:26.233583 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:26.233863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.234726 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.236340 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.236902 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:26.237728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.237728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:26.238460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:26.239477 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.241038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:26.241038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.241038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:26.269992 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:26.270601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:26.270939 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:26.271114 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:26.271847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:26.272169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.273130 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:26.274439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.275347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.275863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:26.276708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:26.277847 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.278619 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.279635 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:26.317857 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:26.318359 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:26.318556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:26.318825 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:26.319642 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:26.320111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.321055 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:26.322398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.323409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:26.325030 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:26.326791 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:26.327499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:26.328479 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:26.329112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:26.329112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:26.329112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:26.329112 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:26.329112 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.329587 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.330507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.331915 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:26.332537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:26.333357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:26.335135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:26.335135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:26.335135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:26.335135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:26.335135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:26.335135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:30.932448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:30.933168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:30.933168 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:30.934174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:30.968529 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:30.968990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:30.970167 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:30.971946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:30.972579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:30.973749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:30.974491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:30.975287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:30.975287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:30.975287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:30.975287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:30.975287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:30.975287 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:31.231692 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:31.232276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:31.232488 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:31.233812 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:31.234118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:31.234279 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.235003 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.236403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.237223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:31.237834 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:31.238580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.239429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:31.239429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:31.239429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.239429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:31.239429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.239429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:31.270677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:31.271302 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:31.271537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:31.271753 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:31.272475 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:31.272968 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.273781 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:31.275353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.275784 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.276753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:31.277105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:31.277445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.277445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:31.277445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.277445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:31.278618 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.279429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.280446 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:31.318452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:31.318858 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:31.318974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:31.319131 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:31.319619 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:31.319993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.320524 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:31.321285 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.322025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.322471 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:31.323034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:31.323673 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:31.324301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.325715 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:31.327918 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:31.328240 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:31.328420 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:31.328628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:31.328996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:31.329233 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:31.329419 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.330072 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.331231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.331732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:31.332293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.332293 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:31.332679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:31.333174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:31.333881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:31.333881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:31.333881 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:35.932674 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:35.933237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:35.933440 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:35.934794 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:35.966713 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:35.967626 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:35.968482 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:35.969426 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:35.970264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:35.970897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:35.971683 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:35.972543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:35.972543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:35.972543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:35.972543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:35.972543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:35.972543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:36.232609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:36.233351 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:36.233567 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:36.234722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:36.235174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:36.235393 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.236322 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.237882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.238632 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:36.239262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.239262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:36.239685 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:36.240273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.242163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:36.242163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.242163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:36.271464 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:36.271995 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:36.272281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:36.272560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:36.273204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:36.273687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.275201 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:36.276645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.277493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.278556 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:36.279150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:36.279702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.279702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:36.279702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.279702 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:36.280808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.281665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.282224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:36.319101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:36.319584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:36.319864 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:36.320151 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:36.320723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:36.321156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.322005 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:36.323439 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.324221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.324654 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:36.325292 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:36.325952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.325952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:36.325952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.325952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.326205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:36.327040 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:36.327983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.327983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:36.327983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.327983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:36.328448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:36.328790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:36.328904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:36.329156 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:36.329428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:36.329647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:36.329783 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.330625 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.333625 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.334555 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:36.335027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.335027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:36.337996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:36.338624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:36.339054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:36.339054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:36.339054 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:40.933689 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:40.934709 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:40.934956 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:40.936393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:40.969805 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:40.970234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:40.971160 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:40.972258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:40.973069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:40.973797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:40.974628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:40.975652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:40.975652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:40.975652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:40.975652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:40.975652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:40.975652 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:41.233494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:41.234096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:41.234347 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:41.235356 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:41.235557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:41.235680 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.236388 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.238133 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.238718 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:41.239494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.239494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:41.240241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:41.241010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.241820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:41.241820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.241820 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:41.272119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:41.273018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:41.273250 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:41.273486 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:41.274201 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:41.274537 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.275436 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:41.277124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.277593 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.279332 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:41.279729 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:41.280104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.280104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:41.280104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.280104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:41.281356 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.281802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.282238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:41.319801 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:41.320349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:41.320562 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:41.320790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:41.321448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:41.321909 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.322843 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:41.324258 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.325158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.325930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:41.326522 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:41.327238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.327238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:41.327238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.327238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.327960 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:41.328697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:41.329223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.329223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:41.329223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.329223 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:41.329223 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:41.329392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:41.329436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:41.329617 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:41.329985 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:41.330245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:41.330424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.330995 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.331944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:41.332704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.332704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:41.333045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:41.333596 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:41.334185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:41.334185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:41.334185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:45.934644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:45.935349 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:45.935428 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:45.936646 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:45.969556 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:45.969756 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:45.970713 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:45.971795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:45.972375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:45.972907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:45.972907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:45.973011 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:45.973905 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:45.974600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:45.974600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:45.974600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:46.234247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:46.234654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:46.235008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:46.236298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:46.236672 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:46.236926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.237873 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.239793 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.240485 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:46.241102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.241102 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:46.242069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:46.244037 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.245882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:46.245882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.245882 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:46.272647 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:46.273100 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:46.273321 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:46.273645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:46.274142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:46.274397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.275109 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:46.276331 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.277017 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.277846 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:46.279089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:46.280624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.281684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.283803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:46.320543 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:46.321016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:46.321153 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:46.321260 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:46.322023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:46.322442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.323283 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:46.324147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.325104 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.325560 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:46.326103 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:46.326663 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:46.327113 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.327592 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:46.329339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:46.329586 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:46.329680 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:46.329873 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:46.330323 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:46.330705 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:46.330967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.331637 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.332467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.333469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:46.334323 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:46.335124 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:46.336128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:46.336128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:46.336128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:46.336128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:46.336128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:46.336128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:50.935318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:50.936048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:50.936048 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:50.937494 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:50.978671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:50.979275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:50.979919 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:50.981460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:50.983484 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:50.983952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:50.985057 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:50.985855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:50.985855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:50.985855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:50.985855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:50.985855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:50.985855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:51.235098 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:51.235513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:51.235733 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:51.236722 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:51.237068 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:51.237312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.238185 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.239620 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.240211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:51.240726 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:51.241319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.242022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:51.242022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:51.242022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.242022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:51.242022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.242022 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:51.273262 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:51.273877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:51.274161 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:51.274409 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:51.275027 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:51.275357 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.276224 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:51.277785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.278789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.279344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:51.280069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:51.280714 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.281543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.282544 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:51.321172 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:51.321677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:51.322007 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:51.322477 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:51.323030 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:51.323360 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.324276 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:51.325957 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.327060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.328034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:51.328755 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:51.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:51.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.329536 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.329720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:51.330403 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:51.331192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:51.331454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:51.331454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:51.331665 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.332386 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.333367 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.333925 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:51.334465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.334465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:51.334781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:51.335273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:51.335785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:51.335785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:51.335785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:55.936096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:55.937305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:55.937305 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:55.938423 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:55.965964 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:55.966386 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:55.967337 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:55.968111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:55.968838 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:55.969479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:55.970245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:55.971253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:55.971253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:55.971253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:55.971253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:55.971253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:55.971253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:56.235599 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:56.236210 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:56.236424 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:56.237546 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:56.238002 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:56.238195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.239125 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.240185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.241079 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:56.241863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.241863 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:56.242148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:56.242868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.243868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:56.243868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.243868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:56.273876 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:56.274796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:56.274796 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:56.275088 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:08:56.275609 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:56.275895 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.276640 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:56.278677 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.279813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.280852 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:56.281300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:56.281845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.281845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:56.281845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.281845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:56.282380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.282897 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.283575 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:56.321925 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:56.322513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:56.322724 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:56.322990 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:56.323623 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:56.324235 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.325130 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:08:56.326486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.327404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.328705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:56.329479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:56.330365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:56.331128 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:08:56.332313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:08:56.332723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:08:56.332723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:08:56.332723 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:56.332847 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:08:56.332954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.333499 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.334430 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.334814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:08:56.335168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.335168 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:08:56.335417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:08:56.335891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:08:56.336380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:08:56.336380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:08:56.336380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:00.936512 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:00.938909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:00.938909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:00.939217 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:00.969481 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:00.969687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:00.970273 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:00.971055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:00.971480 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:00.971886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:00.972398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:00.972951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:00.972951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:00.972951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:00.972951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:00.972951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:00.972951 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:01.236338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:01.236784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:01.237118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:01.238786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:01.239284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:01.239641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.241034 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.242837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.243213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:01.243552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.243552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:01.244749 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:01.245355 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.245885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:01.245885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.245885 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:01.274534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:01.275023 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:01.275251 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:01.275540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:01.276307 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:01.276529 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.277228 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:01.278787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.279595 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:01.280900 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:01.281535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.281535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:01.281535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.281535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:01.282300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.283066 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.283921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:01.322606 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:01.323234 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:01.323450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:01.323685 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:01.324330 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:01.324722 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.325745 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:01.327294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.328028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.329038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:01.329582 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:01.330275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.330275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:01.330275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.330275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.331027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:01.331907 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:01.332813 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:01.333284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:01.333284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:01.333407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.334148 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.334988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.335438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:01.335796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.335796 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:01.336278 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:01.336732 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:01.337284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:01.337284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:01.337284 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:05.938018 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:05.938958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:05.938958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:05.940469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:05.980311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:05.980657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:05.981657 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:05.982728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:05.983298 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:05.983931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:05.984583 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:05.985395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:05.985395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:05.985395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:05.985395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:05.985395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:05.985395 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:06.237104 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:06.237333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:06.237500 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:06.238560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:06.238948 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:06.239195 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.240097 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.241780 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.242398 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:06.243256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.243256 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:06.243752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:06.244804 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.245940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:06.245940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.245940 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:06.275119 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:06.275730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:06.276001 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:06.276450 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:06.276919 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:06.277210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.278494 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:06.280353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.281264 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.282245 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:06.282944 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:06.283589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.284841 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.286031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:06.323246 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:06.323929 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:06.324067 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:06.324328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:06.325281 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:06.325904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.326987 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:06.328384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.329260 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.329713 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:06.330299 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:06.330990 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:06.332095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:06.333171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:06.333855 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:06.334321 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.334321 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:06.334321 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.334321 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.334936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.335532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:06.336492 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:06.337215 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:06.338614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:06.338614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:06.338614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:06.338614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:06.338614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:06.338614 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:10.938699 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:10.939283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:10.939558 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:10.940366 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:10.973596 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:10.974460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:10.975400 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:10.976496 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:10.977100 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:10.977781 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:10.978464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:10.979319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:10.979319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:10.979319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:10.979319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:10.979319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:10.979319 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:11.237967 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:11.238271 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.268025 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.269084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.269926 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:11.270738 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:11.272027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.273187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:11.273187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:11.273187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.273187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:11.273187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.273187 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:11.275662 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:11.276157 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:11.276311 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:11.276501 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:11.277484 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:11.277853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.278786 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:11.279778 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.280486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.280980 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:11.281589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:11.282244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.282244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:11.282244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.282244 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:11.282587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.283186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.283690 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:11.323632 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:11.324192 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:11.324381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:11.324616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:11.325545 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:11.325972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.327196 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:11.328385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.328854 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.330064 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:11.330440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:11.330855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.330855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:11.330855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.330855 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.332853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:11.333538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:11.334243 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:11.334534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:11.334534 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:11.334534 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.334777 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.335708 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.336131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:11.336493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.336493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:11.337029 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:11.337464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:11.337862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:11.337862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:11.337862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:15.939386 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:15.940016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:15.940016 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:15.940690 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:15.976301 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:15.976501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:15.977111 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:15.978007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:15.978450 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:15.978886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:15.978886 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:15.979055 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:15.979742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:15.980589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:15.980589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:15.980589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:16.238238 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:16.238679 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:16.238793 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:16.239318 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:16.239491 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:16.239605 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.240095 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.240916 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.241305 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:16.241640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.241640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:16.241987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:16.242590 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.243094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:16.243094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.243094 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:16.276896 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:16.277123 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:16.277370 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:16.277575 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:16.278203 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:16.278465 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.279259 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:16.281105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.281531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.282946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:16.283309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:16.283697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.283697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:16.283697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.283697 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:16.284955 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.285380 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.285835 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:16.324853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:16.324853 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:16.325101 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:16.325232 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:16.325772 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:16.326132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.326913 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:16.328161 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.328831 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.329571 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:16.330375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:16.331377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:16.332349 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:16.334089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:16.334958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:16.334958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:16.334958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:16.334958 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:16.334958 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.335527 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.336730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.337200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:16.337623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.337623 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:16.338210 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:16.339803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:16.341163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:16.341163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:16.341163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:20.940176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:20.940744 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:20.941015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:20.942519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:20.945448 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG dbcounter [-] [105368] Writing DB stats neutron:SELECT=240 {{(pid=105368) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 05 02:09:20.975276 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:20.975667 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:20.976538 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:20.977267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:20.977701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:20.978101 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:20.978531 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:20.979155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:20.979155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:20.979155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:20.979155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:20.979155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:20.979155 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:21.239095 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:21.239666 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:21.239909 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:21.240908 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:21.241147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:21.241321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.242095 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.243675 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.244377 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:21.245134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.245134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:21.247566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:21.248268 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.248730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:21.248730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.248730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:21.277171 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:21.280176 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:21.280273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:21.280406 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:21.281040 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:21.281427 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.282362 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:21.283280 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.283999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.284424 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:21.284972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:21.285649 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.286385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.287523 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:21.325034 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:21.325598 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:21.325828 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:21.326049 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:21.326660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:21.327019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.327861 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:21.329200 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.329919 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.330808 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:21.331564 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:21.332000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.332000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:21.332000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.332000 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.332600 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:21.333088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:21.333613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.333613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:21.333613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.333613 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:21.333613 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:21.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:21.334185 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:21.334438 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:21.334921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:21.335283 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:21.335460 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.336163 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.336987 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.337322 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:21.337850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.337850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:21.338027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:21.338706 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:21.339388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:21.339388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:21.339388 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:25.940778 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:25.941368 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:25.941560 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:25.942933 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:25.980427 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:25.980704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:25.981402 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:25.982525 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:25.983063 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:25.983597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:25.983597 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:25.983684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:25.984306 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:25.984973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:25.984973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:25.984973 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:26.239730 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:26.240251 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:26.240446 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:26.241170 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:26.241324 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:26.241452 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.242060 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.243177 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.243553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:26.243918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.243918 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:26.244423 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:26.244867 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.245353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:26.245353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.245353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:26.280293 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:26.280834 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:26.281019 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:26.281236 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:26.282003 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:26.282351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.283324 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:26.284731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.285172 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.286224 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:26.286581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:26.286931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.286931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:26.286931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.286931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:26.288354 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.288777 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.289222 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:26.325720 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:26.326292 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:26.326519 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:26.326770 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:26.327447 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:26.327868 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.328744 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:26.330310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.330668 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.331553 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:26.331966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:26.332286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.332286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:26.332286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.332286 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.333080 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:26.333501 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:26.333983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.333983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:26.333983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.333983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:26.334174 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:26.334469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:26.334610 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:26.334829 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:26.335295 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:26.335717 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:26.335930 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.336399 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.337274 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.337599 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:26.338106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.338106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:26.338532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:26.339106 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:26.339528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:26.339528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:26.339528 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:30.941601 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:30.942111 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:30.942212 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:30.943354 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:30.980875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:30.981231 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:30.982062 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:30.982862 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:30.983440 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:30.984027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:30.984723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:30.985550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:30.985550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:30.985550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:30.985550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:30.985550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:30.985550 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:31.240385 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:31.240998 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:31.241120 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:31.242054 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:31.242531 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:31.242891 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.243813 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.244901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.245404 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:31.245887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.245887 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:31.246506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:31.247186 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.247790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:31.247790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.247790 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:31.280860 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:31.281383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:31.281538 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:31.281708 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:31.282566 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:31.282983 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.283889 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:31.284935 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:31.285979 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.286949 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.287688 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:31.288401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.289246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.290132 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:31.326469 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:31.327020 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:31.327134 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:31.327286 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:31.327766 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:31.328115 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.328970 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:31.330327 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.332438 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:31.333019 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:31.333736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.333736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:31.333736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.333736 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.333996 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:31.334795 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:31.335580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:31.336033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:31.336033 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:31.336238 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.336845 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.337861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.338225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:31.338581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.338581 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:31.338924 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:31.339387 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:31.339953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:31.339953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:31.339953 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:35.966922 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:35.968738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:35.968738 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:35.969237 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:36.002444 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.002670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.003552 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.004445 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.004850 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:36.005252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.005252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:36.005829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:36.006371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.007192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:36.007192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.007192 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:36.241138 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:36.241653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:36.241923 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:36.242981 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.243258 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.243498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.244371 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.246146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.247021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:36.247711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.247711 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:36.248109 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:36.248892 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.250049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:36.250049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.250049 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:36.281564 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:36.281979 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:36.282175 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:36.282419 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:36.282988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.283282 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.284151 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:36.285603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.286500 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.287234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:36.287929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:36.288566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.288566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:36.288566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.288566 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:36.289379 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.290120 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.290723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:36.327017 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:36.327509 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:36.327657 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:36.327844 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:36.328313 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.328589 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.329176 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:36.329997 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.330453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.331028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:36.331344 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:36.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:36.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.331704 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.333070 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:36.333521 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:36.334005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.334005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:36.334005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.334005 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:36.335284 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:36.335790 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:36.336015 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:36.336338 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:36.336869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.337105 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:36.337246 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.337728 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.338856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.339185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:36.339498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.339498 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:36.340047 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:36.340723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:36.341792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:36.341792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:36.341792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:40.968687 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:40.969557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:40.969557 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:40.970866 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:41.000257 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.000750 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.001486 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.002283 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.002898 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:41.003548 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:41.004679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.005802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:41.005802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:41.005802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.005802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:41.005802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.005802 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:41.241966 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:41.242328 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:41.242594 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:41.243565 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.243881 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.244150 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.245146 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.247338 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:41.248502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.248502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:41.249263 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:41.250010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.251170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:41.251170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.251170 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:41.282005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:41.282285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:41.282398 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:41.282537 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:41.282977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.283207 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.283776 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:41.284995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.285904 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.286567 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:41.287269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:41.287945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.287945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:41.287945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.287945 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:41.288406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.289193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.290105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:41.327628 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:41.328135 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:41.328530 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:41.328863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:41.329452 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.329893 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.331171 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:41.332799 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.333873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.335908 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:41.336345 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:41.336877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.336877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:41.336877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.336877 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:41.336877 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.339096 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:41.339594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:41.340225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.340518 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:41.340518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.341031 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.341998 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.342347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:41.342707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.342707 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:41.343156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:41.343601 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:41.344189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:41.344189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:41.344189 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:45.969454 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:45.970701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:45.970701 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:45.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:46.038532 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.038753 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.039351 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.040138 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.040803 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:46.041587 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:46.042329 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.043209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:46.043209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:46.043209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.043209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:46.043209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.043209 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:46.242540 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:46.243058 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:46.243247 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:46.244392 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.244725 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.245002 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.246196 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.247963 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.248494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:46.249042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.249042 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:46.250095 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:46.250797 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.251530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:46.251530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.251530 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:46.282837 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:46.283245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:46.283430 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:46.283652 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:46.284211 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.284506 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.285375 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:46.286800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.287466 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.288171 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:46.288490 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:46.288899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.288899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:46.288899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.288899 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:46.289978 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.290409 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.290889 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:46.328228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:46.328751 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:46.329005 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:46.329208 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:46.329869 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.330114 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.330961 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:46.332185 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.333059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.333671 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:46.334252 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:46.335016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.335016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:46.335016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.335016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.335921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:46.336748 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:46.337967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.337967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:46.337967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.337967 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:46.339555 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:46.339903 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:46.340008 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:46.340183 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:46.340653 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.341078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:46.341316 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.342053 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.342931 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.343300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:46.343670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.343670 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:46.343943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:46.344392 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:46.344934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:46.344934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:46.344934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:50.970050 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:50.971164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:50.971164 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:50.972339 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:51.007204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.007473 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.008193 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.009407 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.010069 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:51.010746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.010746 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:51.010972 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:51.011535 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.012144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:51.012144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.012144 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:51.243142 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:51.243587 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:51.243781 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:51.244742 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.244974 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.245088 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.245687 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.247203 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.247856 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:51.248517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.248517 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:51.249045 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:51.249943 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.250493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:51.250493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.250493 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:51.283436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:51.283912 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:51.284102 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:51.284341 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:51.285080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.285444 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.287482 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:51.287751 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.288225 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.289043 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:51.289397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:51.289731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.289731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:51.289731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.289731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:51.293789 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.294218 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.294787 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:51.328932 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:51.329285 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:51.329467 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:51.329671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:51.330326 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.330731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.331689 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:51.333021 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.333413 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.334464 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:51.334837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:51.335206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.335206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:51.335206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.335206 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.336243 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:51.336689 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:51.337202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.337202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:51.337202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.337202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:51.339977 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:51.340350 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:51.340468 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:51.340624 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:51.341115 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.341497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:51.341745 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.342686 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.343719 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:51.344469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.344469 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:51.344932 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:51.345353 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:51.345873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:51.345873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:51.345873 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:55.970718 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:55.971749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:55.971749 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:55.973057 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:56.009579 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.009579 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.009730 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.010482 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.011174 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:56.011720 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:56.012304 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.013038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:56.013038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:56.013038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.013038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:56.013038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.013038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:56.243810 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:56.244333 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:56.244603 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:56.245895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.245895 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.246269 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.247084 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.248630 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.249205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:56.249785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.249785 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:56.250701 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:56.251417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.252247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:56.252247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.252247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:56.284038 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:56.284383 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:56.284669 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:56.284902 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:09:56.285376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.285678 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.286453 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:56.287325 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.288038 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.288532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:56.289169 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:56.290076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.290076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:56.290076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.290076 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:56.290300 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.290999 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.291853 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:56.329456 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:56.329942 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:56.330204 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:56.330426 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:56.331096 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.331385 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.332131 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:09:56.333486 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.334241 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.334993 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:56.335478 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:56.335866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.335866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:56.335866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.335866 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.336134 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:56.336640 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:56.337211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.337211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:56.337211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.337211 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:09:56.340436 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:09:56.340700 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:09:56.340863 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:09:56.341035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:09:56.341578 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.342205 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:09:56.342455 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.343334 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.344267 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.344857 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:09:56.345456 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:09:56.346273 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:09:56.347204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:09:56.347204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:09:56.347204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:09:56.347204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:09:56.347204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:09:56.347204 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:00.971552 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:00.973332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:00.973332 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:00.973739 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:01.010798 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.011289 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.012208 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.013028 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.013669 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:01.014401 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:01.015266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.016309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:01.016309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:01.016309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.016309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:01.016309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.016309 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:01.244637 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:01.245063 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:01.245228 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:01.245970 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.246190 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.246347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.247494 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.248491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.249603 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:01.250312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:01.251020 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.252105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:01.252105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:01.252105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.252105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:01.252105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.252105 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:01.284508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:01.285089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:01.285336 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:01.285544 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:10:01.286218 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.286539 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.287467 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:01.289191 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.289934 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.290532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:01.291162 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:01.291837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.291837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:01.291837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.291837 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:01.292249 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.292816 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.293301 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:01.330028 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:01.330513 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:01.330710 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:01.330952 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:01.331457 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.331806 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.332532 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:01.333858 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.334518 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.335275 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:01.336050 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:01.336724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.336724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:01.336724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.336724 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.336936 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:01.337679 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:01.338546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.338546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:01.338546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.338546 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:01.341035 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:01.342077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:01.342077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:01.342077 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:01.342225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.342584 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:01.342883 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.343770 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.344563 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.344952 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:01.345434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.345434 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:01.345576 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:01.346159 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:01.346647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:01.346647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:01.346647 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:05.972374 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:05.973580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:05.973580 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:05.974660 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:06.004875 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.004875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.005507 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.006152 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.006851 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:06.007737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:06.008634 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.009628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:06.009628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:06.009628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.009628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:06.009628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.009628 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:06.245282 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:06.245784 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:06.246053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:06.247147 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.247478 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.247705 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.248572 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.250127 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.250948 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:06.251447 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:06.252213 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.253262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:06.253262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:06.253262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.253262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:06.253262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.253262 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:06.285127 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:06.285654 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:06.285894 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:06.286225 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:10:06.286983 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.287276 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.288083 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:06.289347 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.290312 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.292365 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:06.292800 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:06.293294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.293294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:06.293294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.293294 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:06.295074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.295742 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.296389 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:06.330616 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:06.331169 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:06.331365 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:06.331572 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:06.332279 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.332684 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.333656 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:06.335010 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.335437 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.336502 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:06.336861 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:06.337234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.337234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:06.337234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.337234 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.338270 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:06.338731 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:06.339266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.339266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:06.339266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.339266 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:06.341497 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:06.341823 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:06.341945 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:06.342113 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:06.342573 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.342988 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:06.343212 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.344091 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.345202 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.345608 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:06.345977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.345977 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:06.346165 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:06.346611 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:06.347135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:06.347135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:06.347135 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:10.973188 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:10.973826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:10.973826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:10.975380 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:11.006091 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.006394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.007240 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.007966 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.008543 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:11.009205 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:11.009988 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.010737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:11.010737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:11.010737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.010737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:11.010737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.010737 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:11.245783 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:11.246987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:11.246987 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:11.247568 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.247826 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.247929 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.248641 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.250073 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.250442 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:11.250814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.250814 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:11.251740 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:11.252199 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.252686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:11.252686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.252686 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:11.285677 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:11.286118 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:11.286300 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:11.286489 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:10:11.287074 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.287384 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.288147 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:11.289310 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.289989 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.290624 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:11.291247 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:11.291917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.291917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:11.291917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.291917 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:11.292296 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.292995 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.293693 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:11.331261 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:11.331627 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:11.331839 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:11.332072 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:11.332550 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.333027 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.334156 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:11.335358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.336052 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.336752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:11.337221 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:11.337829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.337829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:11.337829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.337829 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.338481 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:11.339321 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:11.340090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.340090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:11.340090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.340090 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:11.342089 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:11.342390 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:11.342508 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:11.342675 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:11.343325 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.343921 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:11.344431 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.345315 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.346253 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.346645 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:11.347084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.347084 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:11.347421 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:11.347896 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:11.348594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:11.348594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:11.348594 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:15.973671 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:15.974376 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:15.974510 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:15.975786 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:16.008080 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.008461 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.009394 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.010196 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.010588 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:16.010946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.010946 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:16.011397 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:16.011848 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.012417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:16.012417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.012417 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:16.246381 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:16.246748 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:16.246861 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:16.247811 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-68bc1d51-8ada-4507-8adc-68e5fc8c2562, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.247955 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-bca3048b-ab22-440e-a9f8-31e6bea3944a, routes=[], if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.248059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.248479 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.249158 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.249792 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:16.250125 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:16.250487 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.250954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:16.250954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:16.250954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.250954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:16.250954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.250954 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:16.286298 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:16.286904 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:16.287078 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:16.287273 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting fip-distributed flag in NB_Global to False {{(pid=105368) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 05 02:10:16.287645 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.287901 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.288358 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:16.289545 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.290429 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.290921 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:16.291541 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:16.292406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.292406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:16.292406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.292406 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:16.293153 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                                                           ^^^^^^^^
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.293974 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.295059 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:16.331795 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:16.332245 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_ha_failover {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:16.332421 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:16.332644 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:16.333053 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.333307 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.333875 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 05 02:10:16.335007 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.335752 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.command 
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.336390 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:16.337016 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:16.337723 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:16.338491 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.339662 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics 
May 05 02:10:16.342533 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105368) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 05 02:10:16.342840 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 05 02:10:16.342950 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105368) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 05 02:10:16.343107 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105368) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
May 05 02:10:16.343393 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.343621 np0000171612 neutron-ovn-maintenance-worker[105368]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105368) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 05 02:10:16.343767 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Traceback (most recent call last):
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]:     txn.results.put(txn.do_commit())
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]:                     ^^^^^^^^^^^^^^^
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]:     raise RuntimeError(msg)
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.344188 np0000171612 neutron-ovn-maintenance-worker[105368]: 
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.345065 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.345494 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 05 02:10:16.345903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.345903 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics [None req-2847b085-ede8-4f20-9d94-69ba6bfdcc6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics Traceback (most recent call last):
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     work()
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(*args, **kwargs)
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 05 02:10:16.346565 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     next(self.gen)
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     with self.create_transaction(
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     self.result = self.commit()
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise result.ex
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 05 02:10:16.347074 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 05 02:10:16.347691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics     raise RuntimeError(msg)
May 05 02:10:16.347691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 05 02:10:16.347691 np0000171612 neutron-ovn-maintenance-worker[105368]: ERROR futurist.periodics